Multimethod Assessment And Construct Validation

With the publication of their 1959 article, "Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix," Campbell and Fiske made clear the need to use multiple methods of measurement across multiple traits to evaluate the construct validity of measures.1 Although Campbell and Fiske (1959) proposed several qualitative decision rules for the evaluation of multitrait by multimethod matrices, it was the application of confirmatory factor analysis (CFA) to multitrait by multimethod matrices that provided quantitative procedures to test simultaneously the convergent and discriminant validity of the latent traits as well as the discriminant validity of the latent methods. In addition, the CFA procedures provided information on the amount of trait, method, and error variance in each manifest measure (Eid, Lischetzke, Nussbeck, & Trierweiler, 2003; Lance, Noble, & Scullen, 2002).

The use of CFA to model multitrait by multi-method matrices provides a highly sophisticated set of procedures to determine the construct validity of measures. To exemplify the merits of these procedures, we first describe how CFA can be used to model a multitrait by multimethod matrix. For this

'Our use of the term trait in this chapter is similar to the meaning of the term construct where construct is defined as "a synthetic variable, usually composed of multiple systematically related elements, that is inferred but cannot be directly observed" (Haynes & O'Brien, 2000, p. 297).

example, we use the attention-deficit/hyperactivity disorder (ADHD)-inattention (IN), ADHD-hyperac-tivity/impulsivity (HI), and oppositional defiant disorder (ODD) constructs. We first discuss how the CFA procedures can estimate the convergent and discriminant validity of the individual symptom ratings on a rating scale, thereby allowing the selection of items with good convergent and discriminant validity. We then describe how the procedures can estimate the convergent and discriminant validity of the summary scores for the ADHD-IN, ADHD-HI, and ODD measures. In these examples, we outline the ideal results necessary to establish strong convergent and discriminant validity for the measures. We then outline the less-than-ideal results (i.e., strong method effects) and the complexities that such results create for judging the validity of measures.

After this discussion of CFA, we then review the studies that have used CFA to determine the construct validity of clinical psychology measures. The review of these studies provides information on how well certain constructs in clinical psychology are currently measured (e.g., the convergent and discriminant validity of measures of anxiety and depression in children).

The final section of the chapter offers an expansion of the Campbell and Fiske multitrait by multi-method matrix in terms of multiple types of information—facets (traits), modes, dimensions, instruments, methods, sources, settings, and occasions of measurement. We suggest that this expanded measurement matrix provides a rich conceptual framework to examine the construct validity of measures in clinical psychology (e.g., the determination of estimates of shared variance across the different types of information). This expanded matrix also underscores the issue that the validity of measures in clinical psychology can be conditional (e.g., a measure can be valid for decisions in one setting but not another).

Although our example focuses on ADHD-IN, ADHD-HI, and ODD, it is meant to provide a general framework for estimating the validity of measures for other behavior problems. We thus encourage the reader to substitute his or her favorite three constructs for our three constructs and to work through the example with the alternative constructs.

Was this article helpful?

0 0
The ADHD Success Formula

The ADHD Success Formula

This is an audio and guide that will help you battle through ADHD and Accomplish Twice As Much In Half The Time. Learn more by download your very own copy today.

Get My Free Ebook


Post a comment