Using Multiple Self Report Measures for Construct Validation

It can be stated with some certainty that almost all health psychology research, whether it addresses molecular or broad research questions, involves self-reports of some kind. Reasons for this dominance of self-report measures include that objective measures (e.g., physiological or observational measures) are often not easy to obtain and that for many constructs that health psychologists are interested in, no "objective" indictors exist. For example, no objective measure of self-efficacy is available or currently even conceivable, as the degree of perceived self-efficacy is a subjective phenomenon. Similarly, no reliable objective measures for constructs such as pain or stress exist and, therefore, these constructs are also measured using self-reports or at least self-reports are used as one measure among others.

In many areas of health psychology a multitude of self-report measures has been developed for measuring one and the same construct or related constructs. Prominent examples of constructs for which a large amount of self-report measures exist are stress, self-efficacy, or quality of life. Establishing the validity of these measures or deciding which one is the "best" is made difficult by the fact that often no external criterion of validity exists. In such cases, multiple self-report measures are used for establishing validity and for gaining a better understanding of which aspect of the construct is assessed by a particular self-report measure. A typical example of such a study is Hadorn and Hays' (1991) validation of an instrument to assess health-related quality of life and an instrument for preferences of different health-related quality-of-life states. The authors used multitrait-muldmethod (MTMM) analysis (Campbell & Fiske, 1959) to evaluate the construct validity (convergent and discriminant validity) of the two measures. They used two self-report measurement techniques each to assess health-related quality of life and preferences of different health-related quality-of-life states. As a procedure for implementing the MTMM strategy the authors used confirmatory factor analysis. Their analyses support the construct validity of self-reported health-related quality of life, leading the authors to conclude that either of the two self-report instruments can be used to assess patients' perceived quality of life. On the other hand, they found substantial method variance and little valid trait variance for preferences of different health-related quality-of-life states, a finding that led the authors to replace these measures in future studies.

Another example of this approach is Goldbeck and Schmitz's (2001) study comparing three different generic quality-of-life instruments to examine measurement effects on quality-of-life results in cystic fibrosis patients. The three self-report measures differed in the type of target population for which they were originally developed, the time frame for answering the questions (from the present to the past 4 weeks), and the aspects they address (e.g., well-being and functioning, psychosocial health, physical health). Calculated were internal consistency, convergent and discriminant validity (correlation patterns, common factor analysis), and external validity (correlations with symptom and pulmonary function scores, with intensity of therapy; comparisons with healthy peers) of the three instruments. The analyses revealed comparable reliability (internal consistency) of the three self-report measures, but only partial overlap between them (comparably low interscale correlations), indicating limited convergent validity. Apparently, each questionnaire tapped a slightly different aspect of the construct "quality of life." For example, the social dimension of quality of life is poorly represented in one of the instruments whereas general life satisfaction is poorly represented in another instrument. Both these instruments emphasize more health-related aspects of quality of life. Thus, the analyses revealed in which domain the respective scales perform best. The results of studies such as those just described contribute to a better understanding of the various facets of a theoretical construct and help researchers to choose the instrument(s) that are appropriate for the specific purpose of their study. Although none of the three instruments may cover all dimensions of relevance for describing quality of life in patients with cystic fibrosis, the shortcoming of each individual instrument can be overcome by using them together (Goldbeck & Schmitz, 2001).

A third example is a study by Martin and colleagues (Martin et al., 2000), who assessed construct validity using an adaptation of Campbell and Fiske's (1959) MTMM approach, this time for assessing the convergent and discriminant validity of a migraine-specific quality-of-life questionnaire (MSQ; Jhingran, Osterhaus, Miller, Lee, & Kirch-doerfer, 1998). Specifically, the authors used three analyses to establish construct validity. First, they estimated the MTMM based on a multi-trait-monomethod correlation matrix containing interscale correlations and Cronbach's alpha (internal consistency coefficients) on the diagonal of the correlation matrix. Convergent and discriminant validity were estimated by correlating the MSQ scores with scores from two other self-report measures. The results revealed low to moderate correlations with the other self-report measures, leading the authors to the conclusion that their instrument measures a related, but distinct construct.

Altogether, these examples demonstrate that such studies can assist researchers in identifying the purposes for which a certain self-report measure is suitable. Furthermore, they help in interpreting the divergent results found in studies using different self-report methodologies to measure the same construct. Opposite results may be found if different studies used instruments that emphasize a different dimension or facet of a construct. This is of particular importance when measuring complex phenomena such as stress, where measurement instruments can differ, for example, in their degree of specificity or generality or whether they assess chronic or acute conditions (see Hurrell, Nelson, & Simmons, 1998).

Was this article helpful?

0 0
Naturally Cure Your Headaches

Naturally Cure Your Headaches

Are Headaches Taking Your Life Hostage and Preventing You From Living to Your Fullest Potential? Are you tired of being given the run around by doctors who tell you that your headaches or migraines are psychological or that they have no cause that can be treated? Are you sick of calling in sick because you woke up with a headache so bad that you can barely think or see straight?

Get My Free Ebook

Post a comment