Trait Source and Error Variance

Trait (construct) variance represents the systematic variance in a specific manifest variable associated with a particular latent trait, whereas source variance represents the systematic variance in a specific manifest variable associated with a particular latent source. The error variance in a specific manifest variable involves two different aspects—residual systematic variance (i.e., reliable variance not associated with trait and source factors) and nonsystematic effects (i.e., measurement error; Lance et al., 2002, p. 228). Figure 27.1 shows how the variance in each manifest variable (symptom rating) is separated into trait, source, and error effects.

Trait effects are generally considered to represent systematic variance in the manifest variables that may generalize across sources. Strong trait effects across a set of manifest variables for two or more sources are considered to indicate that the sources view the children's behavior in a similar manner (Greenbaum, Dedrick, Prange, & Friedman, 1994; Rowe & Kandel, 1997). In traditional psychometric theory, a good measure (manifest variable in this context) has a large amount of trait variance and, as will be explained later, a good measure also contains substantially more trait than source variance.

Source effects are usually considered a form of bias associated with characteristics of the rater (Fiske, 1987a). In this view, source effects are considered problematic because they distort or bias the relations among the constructs (Greenbaum et al., 1994). To determine the true relations among a set of constructs, it is considered necessary to remove the source effects from the measures. For example, to determine the unbiased relations among ADHD-IN, ADHD-HI, and ODD, the bias specific to each source must be removed from each measure so that the correlations among the three latent constructs are based on only trait variance.

Source effects can also be considered to reflect meaningful differences in the children's behavior across situations. An example of this view would be a child who shows ADHD-HI behavior in the classroom and does not show such behavior at home. Rather than bias, the mother and father provide a consensual rating for the child's ADHD-HI behavior at home, while the teacher and aide provide a consensual rating for the child's ADHD-HI behavior at school. Instead of the need to eliminate source effects to understand the true relations among ADHD-IN, ADHD-HI, and ODD, source effects can represent meaningful variance (Dishion, Burraston, & Li, 2002). We offer suggestions later in the chapter for how to distinguish between the bias and consensual views of source effects.

Although these are the typical definitions of trait and source effects, these definitions hide a significant complexity. For example, if all the ADHD-HI manifest variables for the parent and teacher sources contain approximately 70% trait variance, then a substantial amount of the variance in the ADHD-HI manifest variables generalizes across the sources, suggesting good convergent validity (as well as discriminant validity because the source effects in each manifest variable have to be less than the trait effects in this example). However, if the ADHD-HI manifest variables for the parent source contain approximately 2% trait and 84% source variance whereas the ADHD-HI manifest variables for the teacher source contain approximately 60% trait and 34% source variance (Burns, Walsh, & Gomez, 2003; Gomez, Burns, Walsh, & Moura, 2003), then the preceding definition of a trait effect runs into a conceptual dilemma (e.g., How can generalization across the parent and teacher sources occur in only one direction?). This conceptual dilemma requires a slighdy different definition of trait variance at the level of a specific manifest variable.

In a multitrait by multisource CFA, trait variance refers to the amount of systematic variance in a specific manifest vari able that is shared with other manifest variables (purportedly representing the same latent trait) rated by the same source assuming negligible source and method variance. If another source or sources rate the same manifest variables, their trait variance is similarly defined, but the amount of trait variance specific to the different sources for the same manifest variables may differ if the different sources impose variations in the ratings that are unique to the specific source. In the case where the amount of trait variance differs substantially between a pair of sources, as does correspondingly the amount of source variance, the accepted definition of trait variance from Campbell and Fiske, or as implied in generalizability theory, may not be appropriate. Here the large amount of source variance for one source may have a meaning and utility corresponding to the traditional definition of trait variance. 0- A. Walsh, personal communication, June 9, 2003)

This more specific definition is needed to deal with the outcome of a large discrepancy in the amount of trait variance across sources for the same manifest variables. We address this complexity later in the chapter. First, however, we will ignore this complexity and describe the ideal set of outcomes for construct validity from the use of CFA to model the multitrait by multisource matrix shown in Figure 27.1.

Was this article helpful?

0 0

Post a comment