Note. The trait, source, and error components sum to 1.0 within rounding error. The specific measures used in the studies can be found in the original articles.
"Adapted from Table 7, "A Multitrait-Multisource Confirmatory Factor Analytic Approach to the Construct validity of ADHD Rating Scales," by R. Gomez, G. L. Burns, J. A. Walsh, and M. A. Moura, 2003, Psychological Assessment, 15, pp. 3-16. Copyright 2003 by the American Psychological Association. ^Adapted from Table 8, "A Multitrait-Multisource Confirmatory Factor Analytic Approach to the Construct validity of ADHD Rating Scales," by R. Gomez, G. L. Burns, J. A. Walsh, and M. A. Moura, 2003, Psychological Assessment, 15, pp. 3-16. Copyright 2003 by the American Psychological Association.
cAdapted from Table 3, "Convergent and Discriminant Validity of Trait and Source Effects in ADHD-Inattention and Hyperactivity/Impulsivity Measures Across a 3-Month Interval," by G. L. Burns, J. A. Walsh, and R. Gomez, 2003, Journal of Abnormal Child Psychology, 31, pp. 529-541. Copyright 2003 by Springer. Adapted with permission.
dAdapted from Table 3, "Relation of Social and Academic Competence to Depressive Symptoms in Childhood," by D. A. Cole, 1990, Journal of Abnormal Psychology, 99, pp. 422-229. Copyright 1990 by the American Psychological Association.
"Adapted from Table 5, "Relation Between Symptoms of Anxiety and Depression in Children: A Multitrait-Multimethod Assessment," by D. A. Cole, R. Truglio, and L. Peeke, 1997, Journal of Consulting and Clinical Psychology, 65, pp. 110-119. Copyright 1997 by the American Psychological Association.
^Adapted from Table 2, "Estimation of Method and Trait Variance in Ratings of Conduct Disorder," by D. M. Fer-gusson and L. J. Horwood, 1989, Journal of Child Psychology and Psychiatry, 30, pp. 365-378. Copyright 1989 by Blackwell. Adapted with permission.
g Adapted from Table 5, "In the Eye of the Beholder? Parental Ratings of Externalizing and Internalizing Symptoms," by D. C. Rowe and D. Kandel, 1997, Journal of Abnormal Child Psychology, 25, pp. 265-275. Copyright 1997 by Springer. Adapted with permission.
''Adapted from Figure 1, "The Factorial Structure and Construct Validity of the Psychopathy Checklist-Revised (PCL-R) Among Alcoholic Inpatients," by M. Windle and L. Dumenci, 1999, Structural Equation Modeling, 6, pp. 372-393. Copyright 1999 by Erlbaum. Adapted with permission.
'Adapted from Table 4, "Multitrait-Multimethod Model of Adolescent Deviance, Drug Use, Academic, and Sexual Behaviors," by E. A. Tildesley, H. Hops, D. Ary, and J. A. Andrews, 1995, Journal of Psychopathology and Behavioral Assessment, 17, pp. 185-215. Copyright 1995 by Springer. Adapted with permission. ^Adapted from Table 4, "Validity of Self-Reports of Alcohol and Other Drug Use: A Multitrait-Multimethod Assessment," by A. W. Stacy, K. E Widaman, R. Hays, and M. R. DiMatteo, 1985, Journal of Personality and Social Psychology, 49, pp. 219-232. Copyright 1985 by the American Psychological Association. Adapted from Table 5, "Construct Validity of Dimensions of Adaptive Behavior: A Multitrait-Multimethod Evaluation," by K. F Widaman, A. W. Stacy, and S. A. Borthwick-Duffy, 1993, American Journal of Mental Retardation, 98, pp. 219-234. Copyright 1993 by the American Association of Mental Retardation. Adapted with permission.
ness, manipulativeness, lack of empathy) and behavioral (e.g., early onset of criminal behavior, impulsivity) dimensions of antisocial personality disorder (Windle & Dumenci, 1999).
Only three studies contained measures that consistently showed more trait than source variance. Two of these studies focused on measures of alcohol, cigarette, and marijuana use (Stacy, Widaman, Hays, & DiMatteo, 1985; Tildesley, Hops, Ary, & Andrews, 1995). One possibility for this outcome may be the specificity of measures of drug use relative to the measures of anxiety, depression, conduct problems, and ADHD (e.g., the concreteness of constructs, see also Doty & Glick, 1998, pp. 380-381; Haynes & O'Brien, 2000, pp. 128-139). The third study focused on staff measures of competence in adults with a diagnosis of mental retardation (Widaman, Stacy, & Borthwick-Duffy, 1993). Part of the reason for the large amount of trait variance in this study may be the careful attention paid to the development of the measures prior to the CFA.
An additional study appeared in a recent book chapter (Dishion et al., 2002). Here the focus was to estimate the trait, source, and error variance in measures of parenting competence (i.e., monitoring, limit setting, positive reinforcement, relationship quality, and problem solving). The three sources were parents, adolescents, and staff. Nearly all the measures contained more source than trait variance. The authors also reported that the source effects predicted authority conflict and drug use with these correlations being stronger at times than the correlations of trait effects with authority conflict and drug use. These correlations suggest that the source effects in this study contained meaningful variance rather than only bias.
With the exception of the two studies that focused on alcohol/drug use and one study where the focus was on adjustment in adults with a diagnosis of mental retardation, all the other studies indicated that source effects were stronger than trait effects. Several recommendations stem from these results. First, clinical research and clinical decisions should probably never occur on the basis of a single source because of the pervasive nature of source effects. Second, given that the amount of trait variance appears to increase when measures have a higher level of specificity (e.g., Doty & Glick, 1998), it may be possible to develop measures with larger amounts of trait variance with more careful attention to content validity (Haynes et al., 1995). Third, the use of CFA to model multitrait by multi-source matrices should be mandatory in the latter stages of the validation of multisource rating scales. And, finally, research should begin to clarify the meaning of source effects. If source effects remain strong even with more careful attention to the development of more specific measures, then it becomes increasingly important to understand these effects (e.g., Dishion et al., 2002).
The development of measures in clinical psychology could also benefit from a broader framework than the traditional multitrait by multimethod matrix. In this final section, we describe how the matrix can be expanded to include additional types of information.
MULTIPLE TYPES OF INFORMATION: EXPANDING THE MULTITRAIT-MULTIMETHOD MATRIX
The types of information relevant to the development of measures in clinical psychology include facets, modes, dimensions, instruments, methods, sources, occasions, and settings (Haynes & O'Brien, 2000). Although it is not practical to use CFA to model the complete matrix in a single study, CFA can model different aspects of the matrix dependent on the specific goals and stage of measure development (e.g., a multifacet by multisource study; a multifacet by multimethod by multisource by multi-occasion study). In addition, we wish to emphasize that each of these types of information represent a potentially significant cause of variability in measures of clinical phenomena and to understand clinical phenomena, the development of measures needs to examine these influences in a systematic manner.
Was this article helpful?