Multimethod thinking in psychological assessment was influenced most strongly by the seminal paper of Campbell and Fiske (1959). No other publication so importantly shaped researchers' awareness of the crucial role multimethod designs play in the construction and validation of measurement instruments (Shrout & Fiske, 1995). Although Campbell and Fiske (1959) did not make reference to Brunswik's work, their proposals were guided by similar insights and ideas. Campbell and Fiske
(1959) introduced the multitrait-multimethod matrix, a flexible, conceptual and methodological framework for the examination of convergent and discriminant validity. The MTMM matrix is a matrix of correlations among tests. Tests are trait-method units. An MTMM matrix is usually derived from a three-dimensional raw data box consisting of a person facet, a facet of attributes (traits), and a method facet. Although not commonly done, the general MTMM idea could be applied to any other combination of three dimensions of the data box. Instead of measuring traits of persons, properties of stimuli could be measured with different methods and submitted to an MTMM analysis.
For obtaining the most common type of an MTMM matrix, two or more traits (of several individuals) must be measured with two or more methods. The matrix contains four kinds of correlations (see Table 2.1). The elements in the main diagonal are called monotrait-monomethod (mTmM) correlations. They compose the reliabilities of the tests (trait-method units). Correlations among different traits measured with the same method are hetero-trait-monomethod (hTmM) correlations. Correlations among different methods for the same trait are termed monotrait-heteromethod (mThM) correlations. Finally, correlations among different traits that were measured with different methods are named heterotrait-heteromethod (hThM) correlations.
Correlations among different methods for the same trait (mThM) display convergent validity. These correlations should be high. Correlations among different methods for different traits (hThM) are usually the lowest correlations in an MTMM matrix because these tests have neither traits nor methods in common. However, hThM correlations differ from zero if traits or methods are correlated. A self-report measure (Method 1) of Trait A may be correlated with a peer-rating measure (Method 2) of Trait B because A and B are correlated. In addition, both measures may share method variance. Common method variance may be caused by individual differences in self-presentational concerns. If both traits are socially desirable, individuals will differ regarding how favorably they present themselves in the self-report measure of Trait A. Individuals may also differ regarding how
favorably they present themselves to peers. As a consequence, peer-ratings of Trait B will also be affected by self-presentational concerns, and both measures will correlate even if A and B are independent traits. Evidently, correlations among tests are only inflated if the correlations among the traits and the methods have the same sign. If correlations among traits differ in sign from correlations among methods, they may cancel each other out, resulting in low or zero hThM correlations, even if the traits are correlated.
Correlations among different traits measured with the same method (hTmM) ideally should not exceed correlations among different traits measured with different methods (hThM). Such an ideal pattern suggests that using the same method for different traits does not inflate the correlations among the tests because of the use of the same method. For the same reason, correlations among different methods for the same trait (mThM; convergent validity) should ideally not be lower than the reliabilities of the tests (mTmM). Again, such an ideal pattern suggests that the reliabilities of the tests are only because of trait variance but not to shared method variance. The last two comparisons (hTmM versus hThM; mThM versus mTmM) provide estimates for the discriminant validity of tests. Tests display discriminant validity if they do not measure traits they should not. In an MTMM analysis, discriminant validity is achieved when the reliability of a test was not inflated compared to its convergent validity and
Was this article helpful?