The Campbell-Fiske guidelines are frequently criticized for being based on correlations among observed variables rather than among latent constructs. Ironically, in the typical CFA MTMM approach, a single scale score—often an average of multiple items—is used to represent each trait-method combination. Marsh (1993b; Marsh & Hocevar, 1988), however, argued that it is stronger to incorporate the multiple indicators explicitly into the MTMM design. When multiple indicators are used to represent each scale, CFAs at the item level result in a MTMM matrix of latent correlations, thereby eliminating many of the objections to the Campbell-Fiske guidelines. Furthermore, CFA MTMM models can be applied to the latent MTMM matrix in much the same way as they are applied to correlations among measured variables. For example, when a first-order factor is defined by multiple indicators of each trait-method combination, trait and method factors can be represented as second-order factors. This multiple indicator approach also provides a rigorous test of the a priori factor structure used to construct scale scores that is typically untested in the traditional MTMM approaches. With this approach, researchers can separate measurement error that is due to lack of agreement among multiple items from residual variance that is unexplained by trait and method effects. Marsh (1993b) demonstrated this multiple indicator approach for 4 self-concept scales measured on each of four occasions (a 4 scale x 4 time MTMM design) using multiple indicators of each of the 16 (4 x 4) trait-method combinations.
Marsh, Richards, Johnson, Roche, and Tremayne (1994) demonstrated an interesting variation of this MTMM approach and the importance of attending to the item level in a study of one new and two existing physical self-concept instruments. They began with a content analysis of items and classified scales from the three instruments as matching, partially matching, or nonmatching. Treating the extent of "matchingness" as having at least three categories was an important concession in that existing measures typically do not consist of parallel scales as is implicit in the traditional MTMM application. They initially conducted a large CFA based on multiple indicators to represent the 11,5, and 7 a priori factors from the three instruments. They then applied the traditional Campbell-Fiske criteria to their latent MTMM (23 X 23 correlation) matrix resulting from their CFA, emphasizing that inferences were based on latent correlations based on their measurement model relating multiple indicators and latent factors. Based on the a priori predictions derived from their content analysis, the 167 correlations between the 23 latent constructs representing different instruments were classified into 3 a priori categories: 9 convergent validities in which the scales were most closely matched (.79 to .90; median r = .84), 6 convergent validities in which the scales were less closely matched (.61 to .73; median r = .68), and the remaining 152 correlations among nonmatching constructs (.02 to .74; median r = .44). In support of construct validity— and the usefulness of the two categories of matchingness—correlations in the first category were systematically larger than those in the second category. There was also good support for discriminant validity in that the remaining 152 correlations were smaller than convergent validities.
Was this article helpful?