Four outcomes are required for the individual symptom ratings to have strong construct validity. First, each symptom rating in Figure 27.1 must have a substantial loading on the appropriate trait factor (i.e., each ADHD-IN symptom rating has a substantial loading on ADHD-IN trait, each ADHD-HI symptom rating has a substantial loading on the ADHD-HI trait, and each ODD symptom rating has a substantial loading on the ODD trait). Such outcomes would provide support for the convergent validity of each symptom rating. However, even though each symptom rating has a large loading on the appropriate factor, such does not provide information on the discriminant validity of the ratings. The second outcome addresses this issue.
Second, for the symptom ratings to demonstrate discriminant validity (i.e., more trait than source variance), each symptom rating is required to have a much stronger loading on the appropriate trait factor than on its respective source factor (i.e., each ADHD-IN symptom rating has a stronger loading on the ADHD-IN trait factor than on its source factor; each ADHD-HI symptom rating has a stronger loading on the ADHD-HI trait factor than on its source factor; and each ODD symptom rating has a stronger loading on the ODD trait factor than on its source factor). If such outcomes occurred for the all the symptom ratings for each source, such outcomes would provide good support for the convergent and discriminant validity of the symptom ratings.
The third outcome to consider involves the correlations among the ADHD-IN, ADHD-HI, and ODD latent traits. Here there should be evidence for the discriminant validity of the three traits (i.e., the correlations among the latent traits are not too high). Discriminant validity among the traits is important because such is a prerequisite for research that attempts to identify unique attributes for each trait (e.g., unique causes, risk factors, associated features, outcomes, treatment responses and so on). For example, if the correlation between the ADHD-HI and ODD traits was higher than .90, it would be difficult to identify unique features predictive of each trait due to the small amount of unique variance in each. Such a high correlation would also suggest that ADHD-HI and ODD, as measured with these instruments and sources, did not represent separate traits.
The fourth outcome concerns the correlations among the latent source factors. Here the correlations among the mother, father, teacher, and aide source factors must also show discriminant validity (i.e., latent source correlations that are not too high). This relates to the discriminability of the sources, and very high correlations among the sources would suggest a problem of common source bias.
Of these four outcomes, outcomes 1 and 2 are the most central. If these two outcomes occur, then each symptom rating has convergent and discriminant validity. These outcomes are important because it is the amount of trait and source variance in the individual symptom ratings that determines the meaningfulness of the discriminant validity results for the latent traits and sources. For example, if the ADHD-IN, ADHD-HI, and ODD symptom ratings contained an average of 5% trait variance across the four sources, then small correlations among the three traits would not be very meaningful. In contrast, if the average amount of trait variance in the symptom ratings was 70%, small correlations among the three traits would provide good evidence for discriminant validity. The same logic applies to the interpretation of the correlations among the latent source factors.
If these outcomes occurred, then each symptom rating on this particular rating scale would have demonstrated strong convergent and discriminant validity for the four sources (i.e., the construct validity is conditional on this particular scale and these four sources). In addition, each of the three latent traits would have demonstrated strong convergent and discriminant validity. This evidence would therefore indicate that this particular rating scale provided a good measure of these three traits across the four sources. To our knowledge, there is no study in clinical psychology that has used these procedures to evaluate and select the final set of items for a multisource rating scale (e.g., out of approximately 2,000 manuscripts submitted to Psychological Assessment from 1998 to 2003, none used this approach).
Was this article helpful?