Edgar Erdjelder and Jochen Musch
What can be gained from applying methods of experimental psychology to problems of psychological assessment? Experimental psychology and psychological assessment, although both being important branches of psychology, are clearly distinct scientific disciplines with unique histories, characterized by discipline-specific theories, paradigms, and research methods (see Bringmann, Lück, Miller, & Early, 1997). Consequently, there has been little overlap between research in experimental psychology and research in psychological assessment. Despite some influential attempts at bringing the disciplines more closely together (e.g., Cronbach, 1957), little has been said about how the two disciplines could profit from each other. In this chapter, we aim at closing this gap by describing and illustrating benefits that can be gained from applying experimental methods to problems of psychological assessment.
Our chapter comprises seven sections. In the first section, we look at the origins of the experimental method in psychological research, describe its characteristics, and present a definition of the term psychological experiment. We then show in section two that if the term experimental has been used in the context of psychological assessment, it typically has been associated with meanings different from that in experimental psychology. We argue that it is both possible and useful to redefine experimental assessment methods in a way that is consistent with the notion of a psychological experiment in experimental psychology. In sections three and four, major benefits and potential problems of the experimental approach to psychological assessment are discussed.
Sections five and six are devoted to illustrating the benefits of experimental assessment methods and ways to overcome the problems associated with this approach. In section five, we describe experimental methods of assessing the truth in cases in which people are motivated to conceal this truth. Four methods will be critically discussed: the randomized response technique (RRT), the unmatched count technique (UCT), the control question technique (CQT), and the guilty knowledge test (GKT). Consistent with the multimethod approach to psychological measurement, these techniques adhere to decidedly different rationales to overcome response bias in sensitive areas. The RRT and the UCT are techniques to ensure the anonymity of respondents in surveys. They aim at encouraging more honest responding on a voluntary basis and use an experimental between-subject manipulation of question content. The CQT and the GKT are the most frequently used methods of polygraph lie detection and coercively try to determine the guilt or innocence of suspects who deny liability for a crime. Both the CQT and the GKT are based on a within-subject manipulation of question content. It will soon become clear why we classify these four approaches as truly experimental, whereas related methods of assessing the truth that are often given
The work on this chapter has been supported in part by grants from the TransCoop Program of the Alexander von Humboldt Foundation and the Otto Selz Institute, University of Mannheim, Germany the label "experimental" as well (e.g., the bogus pipeline procedure; Jones & Sigall, 1971) do not qualify as experimental assessment methods in the sense defined here and, consequently, will not be discussed further in the present chapter.
In survey research, the true answer to a question is often better conceived of as lying on a continuous rather than a dichotomous scale. Section six addresses the problem of systematic errors in rating-scale or multiple-choice assessments induced, for example, by the order in which questions are arranged or by the range of response options offered to the respondent. We summarize how to cope with these problems using methods of experimental psychology. The concluding seventh section presents guidelines on how problems of psychological measurement can be solved effectively using experimental methods. Moreover, we present a list of criteria that should be met whenever experimental assessment methods are to be applied in practice.
Was this article helpful?