Benefits of Experimental Assessment

What are the benefits of the experimental approach to psychological assessment? First, and perhaps most important, the experimental method provides sound solutions to what we have called decomposition problems elsewhere (e.g., Buchner, Erdfelder, & Vaterrodt-Plunnecke, 1995; Erdfelder & Buchner, 1998a, 1998b). Decomposition problems arise because empirical psychological variables (such as test scores, response times, etc.) are almost always affected by more than a single psychological state or trait. In general, therefore, the observed scores cannot be regarded as "trait-pure" or "state-pure" measures of specific psychological constructs. Rather, the empirical measures are better conceived of as composites of different psychological constructs, each contributing to the observed scores in an uncontrollable manner. For example, as we have seen earlier, the number X. of Shepard-Metzler problems that can be solved in a prespecified time frame (e.g., 3 minutes) is a composite of a sensorimotor component aj and a mental rotation speed component bv

A decomposition problem can be defined as the problem of finding a method that provides pure measures of the psychological constructs of interest, uncontaminated by other psychological traits or states that are involved in the task. In other words, decomposition problems are problems of maximizing the validity of psychological assessment by decomposing the observed test scores into pure measures of the to-be-assessed constructs. A decomposition problem can be solved if an identifiable measurement model is available that essentially defines a one-to-one mapping of parameters characterizing the to-be-assessed psychological constructs ("latent variables") on parameters characterizing the distribution of the empirical variables observed in the test situation ("manifest variables"). Data obtained under single testing conditions are usually insufficient for defining measurement models that are both psychologically plausible and identifiable. As we have illustrated using the Shepard-Metzler task, experimental methods can often help in such situations by enriching the empirical domain of the assessment paradigm. We will further illustrate this point in the next section by showing how responses in interviews can be decomposed, roughly speaking, into true responses and a social desirability component.

A second benefit of experimental assessment methods is closely related to the first. When psychologists assess individual or group characteristics in applied settings, they often aim to explain a particular state of affairs, for example, failure at school, phobia symptoms, or memory problems (e.g., Westmeyer, 1972). What does it mean to explain human behavior scientifically? Since the pioneering work of Hempel and Oppenheim (1948), scientific explanations are generally conceived of as logically correct answers to "why" questions such as "Why does my son fail at school?" or "Why does my daughter suffer from a spider phobia?" The description of the to-be-explained state of affairs is called the "explanandum," and the sentences from which it is logically deduced are called the "explanans." According to Hempel and Oppenheim (1948), the explanans always consists of at least one empirically well-established general law and at least one empirically verifiable antecedent condition that together imply the explanandum. For example, if E denotes the explanandum, then a permissible explanation of E might be an argument of the following modus ponens structure:

A1 (antecedent condition 1)

A2 (antecedent condition 2)

Therefore: E (explanandum)

The goal of experimental psychology is to develop and to test general laws that can be used for scientific explanation (e.g., Bredenkamp, 2001). In contrast, psychological assessments aim to show that the antecedent conditions necessary for deducing the explanandum from the general laws are in fact met. To stick with the preceding example, assessments investigate whether A1 and A2 are indeed true for a particular individual or group to which the explanandum refers (Westmeyer, 1972). Unfortunately, however, the constructs measured by standard clinical or educational tests very often do not match any of the psychological constructs or processes involved in the laws of experimental psychology. Therefore, the classical Hempel-Oppenheim schema of scientific explanation, although accepted by many psychologists, is useless unless a solution is found to this "correspondence problem" between theories of experimental psychology on the one hand and psychological assessment methodology on the other hand. Obviously, by directly referring to particular laws and models of human behavior, experimental assessment methods show a way to address this problem. Thus, experimental measures provide a means of explaining behavior in the strict sense defined by Hempel and Oppenheim (1948).

A third benefit is more measurement-theoretic in nature. Psychological assessment methodology is often plagued by the problem of meaningfulness (e.g., Suppes & Zinnes, 1963). In representational measurement theory, a proposition about measurement results is called "meaningful" if a permissible rescaling of the measures never changes the truth value of the proposition. For example, the sentence "the measures X and Y correlate by r = .80" is meaningful if only if X and Y are interval scales with respect to the underlying constructs they supposedly measure. If this assumption is true, then the measures are unique up to linear increasing transformations, leaving Pearson correlations unaffected. However, if they are actually ordinal rather than interval scales, then the scale values may be subjected to any monotonically increasing transformation, and this can affect Pearson correlations more or less drastically. Obviously, any interpretation of Pearson correlations presumes, either implicitly (e.g., in most validity studies) or explic itly (e.g., in structural equation modeling), that the measures involved are interval scales with respect to the underlying constructs.

The key problem with assumptions on measurement scales is that there is often neither a way of testing them directly nor a way of proving their truth mathematically (Aiken, 1999, p. 40). Again, experimental assessment offers a way to address this problem. Because the measures derived from these methods are, by definition, components of psychological laws or measurement models, their scale properties can often be analyzed mathematically by investigating the structure of the laws that define these measures.

To illustrate, let us again consider the linear law of mental rotation. It relates two physical quantities, the angle of rotation (in degrees) and the response time (in milliseconds), both of which are ratio scales with respect to the physical dimensions of rotation angle and response time, respectively. We could thus arbitrarily decide to use other units of measurement for both scales. If c' denotes the multiplicative scale factor implied by changing the units of measurement for the rotation angle, and if c" represents the corresponding scale factor for time, Equation (3) would change as follows:

Thus, a permissible scale transformation of the physical variables induces a linear transformation of the type v' = (c' / c") v. on the psychological scale values measuring mental rotation speed. We may conclude that v; measures the mental rotation speed on a ratio scale, provided that the linear law of mental rotation is indeed valid.

Was this article helpful?

0 0

Post a comment