Despite its vast impact on multimethod thinking in psychological assessment, the Campbell and Fiske (1959) strategy of comparing correlations suffers from a number of shortcomings of which Campbell and Fiske had been aware without offering satisfactory solutions. First, no statistical test exists for evaluating the pattern of differences among the correlations of the MTMM matrix simultaneously. Second, no straightforward decomposition of test variance into trait variance and method variance can be obtained from the comparisons among correlations suggested by Campbell and Fiske. Third, the comparison of correlations does not consider differences in test reliability and other factors affecting the magnitude of correlations.
Joreskog (1969) presented the most elegant solution to these problems with his general CFA approach to confirmatory factor analysis. Since Joreskog's seminal contribution, MTMM research has shifted from the mere description of correlations to modeling the covariance structure among trait-method units. This methodology has several advantages over the descriptive comparison among correlations. First, by modeling traits and methods as latent variables, reliability differences between tests can be handled. Second, models can be tested and different models can be tested against each other if they are nested (Widaman, 1985). Third, the variance of tests can be decomposed into proportions due to trait factors, method factors, and measurement error. Fourth, if the time facet is included in addition to the person, construct, and method facets, latent state-trait method analyses can be performed, and the variances of tests can be decomposed into proportions due to traits, occasion-specific effects of the situation, methods, and random measurement error (Kenny & Zautra, 2001; Schmitt & Steyer, 1993). Given these advantages of confirmatory factor analyses, it is not surprising that Marsh (1989) counted twenty different CFA-MTMM models 20 years after Joreskog's paper.
Widaman (1985) proposed a taxonomy for many of these models by cross-classifying four trait structures with four method structures. Both structures differ in the number of common factors and whether these factors are orthogonal or oblique. The taxonomy generates a family of 16 hierarchically nested models with the null model (no trait factor, no method factor) being the most restrictive model and the correlated-traits-correlated-methods model (CTCM) being the least restrictive model. Widaman's (1985) taxonomy holds value because it is more systematic than any earlier proposal, provides a heuristic for conceptualizing alternative trait and method structures, and serves as a guideline for testing models sequentially.
Although the general CFA approach has advanced MTMM research tremendously, it has limits. Improper solutions are a common problem with the popular CTCM model. Iterative procedures for estimating parameters of this model often fail to converge or lead to estimates outside the permissible range with a negative variance of one of the method factors being the most frequent problem. Solutions for overcoming this and other problems (e.g., Eid, 2000; Kenny & Kashy, 1992; Marsh, 1989) will be discussed in chapter 20.
Was this article helpful?