Psychological measurement studies traditionally focus only on internal consistency measures of reliability—the extent of agreement among multiple items designed to infer the same construct. Although items are an important facet of measurement error, there are other sources of measurement error such as time (as in test-retest stability approaches to reliability). Thus, studies that ignore other sources of unreliability provide inflated reliability estimates. Can multiple facets of measurement error be modeled simultaneously within the same study?
Marsh and Grayson (1994a) extended the logic of MTMM analyses to address this issue for responses to 6 self-esteem items collected on 4 occasions (a 6 item x 4 occasion design). This is an interesting extension of the traditional MTMM design in that both facets (items and time) represent what are typically considered to be method facets, and there were no multiple trait factors. Starting with the classical measurement theory and extending the logic of MTMM analyses, Marsh and Grayson (1994a) developed SEM models to partition variance into common factor, time-specific, item-specific, and residual components. They emphasized items and time as sources of measurement error used to assess reliability, but outlined how their approach could easily be expanded to include additional facets (e.g., the use of multiple markers when evaluating essays so that there would be time-specific, item-specific, marker-specific, and residual components of error). Although they considered only a single self-concept factor, their approach could also be extended to include multiple traits like those traditionally emphasized in self-concept research. Whereas Marsh and Grayson developed their models from the perspective of SEM, analogous developments have been incorporated into generalizability theory and its focus on validity generalizability (see Schmidt & Hunter, 1996; Shavelson & Webb, 1991).
Was this article helpful?