A continuous conception of nonreactivity assumes it to be an ideal state rather than a feature inherent in certain techniques of measurement. Here, measures are assumed to vary in the extent to which they come close to this ideal. According to this notion, the reactivity of a measure increases with the extent to which both the research subject and the researcher are involved in the act of measurement. We will now introduce six types of measures (see
Table 14.1) that vary in the extent to which nonreactivity is realized, starting with the Type 5 measures that exhibit the highest degree of nonreactivity.
Type 5. This type of data is generated without any initial intention of measurement on the part of the researcher and, consequently, is also collected without any awareness of such an intention on part of the participant, thus representing the most nonreactive and unobtrusive method that can be thought of (Table 14.1, Type 5). Examples of such measures are the various natural accretion and erosion measures (Webb et al., 1966) referring to material and settings not placed or designed for research purposes. These measures use physical "traces" of behavior that, for example, might manifest themselves as remnants like garbage that can be interpreted as an indicator of certain lifestyles, or they can be found in erosion phenomena such as in the paths pedestrians make in the snow showing their preferred routes. Another example of a Type 5 measure (Table 14.1) is the content analysis (see Lubinski, this volume, chap. 8) of archival material. This may be considered a nonreactive measure if there is a guarantee that the records represent natural behavior or if, at the very least, the researchers are aware of the conditions and original aims of potentially reactive data generation in the past. They have to make sure that those specific circumstances of data generation have not distorted measurement in a way that could influence the results of their present investigations. Accordingly, as a database for the measurement of an individual's health status, for instance, using archival self-report data on individual absenteeism should be less reliable than analyzing the individual costs statistics from a health insurance company. Although such analyses are not immune to the researchers' interpretational bias, the mere generation of the data is indeed free of researcher bias.
Type 4. Table 14.1 specifies additional types of measurement that represent differing lesser degrees of nonreactivity. Type 4 measures do not make use of naturally occurring records, but rather measure behavior in a deliberately selected setting or even create opportunities for subjects to behave in a certain way. This behavior is a top-down operational-ization of the variables the researcher is interested
Continuum of Nonreactivity in Measurement
Level of nonreactivity
_TypeO_Type 1_Type 2_Type 3_Type 4 Type 5
Setting initially designed or yes yes yes yes yes no selected for research_
Participants are aware of likely likely likely likely no no the research setting_
Participants are aware of likely likely likely no no no the research question_
Participants are aware of likely likely no no no no the research hypothesis_
Participants are aware of the measures' manipulability likely_no_no_no_no_no_
Examples Participative Bogus pipeline Personality Cover story Lost letter Analysis of expert interviews technique questionnaires experiments technique archival data in and has hypotheses about. However, in this type of measure, subjects are not aware of being the object of measurement. Examples of Type 4 techniques include hidden observation, many of the controlled accretion and erosion measures (Webb et al., 1966; see also later in this chapter), as well as the lost letter technique mentioned earlier. In the latter method, for example, a behavior is recorded that would not have occurred naturally and without the intervention of the researchers, simply because the letter addressed to the mosque would not have been found lying in front of a mailbox. We have previously discussed possible experimenter effects in applying the lost letter technique that might be transmitted by the active design of the situation. Of course, Type 4 measures also have important advantages-over the use of already existing records. If, for example, researchers are interested in inferences of causal relationships between predefined variables, they must be able to manipulate specific features of the subject's environment to ascribe differences in the dependent measure to the work of specific experimental conditions (see also Erfelder &
Musch, this volume, chap. 15). Take, for instance, the director of an arts museum who assumes that children are more attracted to colored pictures than adults and that adults, as opposed to children, are more interested in looking at pictures of high unconventionality. To test both hypotheses in a 2 (picture in color vs. black and white) x 2 (picture of high vs. low unconventionality) X 2 (adults vs. children) factorial design, the director could first equip one exhibition room with pictures representing combinations of the respective conditions (color and high unconventionality, color and low unconventionality, black/white and high unconventionality, black/white and low unconventionality). In a second step, the director would measure the amount of attention paid to each picture and whether children or adults are paying the attention. Following Webb et al. (1966), measuring carpet erosion in front of each picture would also provide an estimate of relative attractiveness. Because of the need to differentiate between adult and children visitors, the underlying material should be sensitive enough to depict the size of shoes. Hence, the director decides that a high and soft flooring is necessary. The director instructs an assistant to vacuum the room every 30 minutes and to count the differently sized footprints before vacuuming. This example provides an obvious demonstration of how Type 4 measures can offer additional opportunities for a more purposeful and controlled measurement than simply using preexisting records. This method is often more efficient and sometimes even the only way that a specific research question can be answered in a nearly nonreactive manner.
Type 3. In measures of Type 3, not only is the research setting prepared by the researcher, but the research context is also not intended to be unobtrusive to the participants. Thus, they are very likely to be aware of participating in a purposeful study. However, it is important to note that the participants' general knowledge about the research context does not necessarily imply their knowledge about the research aims. In fact, Type 3 participants are not informed about or are even actively hindered from elaborating their own assumptions about the research question. As people are usually very interested in knowing about the aim of the research they are participating in, withholding initial information about hypotheses is a necessary feature of most Type 3 methods. Social psychological experiments, for example, often involve active deception of research participants to avoid distortions that can often be traced back to social desirability concerns (for an overview of different methods, see Aronson, Ellsworth, Carlsmith, & Gonzales, 1990). For instance, in an experimental study on the impact of justifications and excuses on the violation of proenvironmental norms, Fritsche's (2003) aim was to manipulate the accessibility of valid accounts prior to the measurement of the norm violating behavior. Because it could be expected that people who are informed about the research topic are both willing and able to influence the dependent measure in the direction of (or contrary to) their own hypotheses (in this case on the relationship between account-giving and socially appropriate behavior), to prevent the generation of participants' own hypotheses, this study was introduced as an investigation of communication over the Internet. The instructions and procedures made this plausible because participants were asked to interview an anonymous partner in a chat room about apparently randomized pairs of topics including "environmental protection" and "guilty conscience." Shielded by this cover story, the confederate chat partner was able to present standardized justifications and excuses in the course of "natural communication" without revealing the actual research question. The experimenter told the participants that they could order a drink, which they would receive later during the experiment. After the manipulation of a specific account's accessibility (i.e., its mere introduction by the confederate) and validity (i.e., its evaluation as situationally appropriate by the confederate) the drinks (in aluminum cans!) were brought in. A previous study with a comparable sample had shown that drinking from cans was perceived as harmful to the environment. Whereas nearly all (92%) of the participants who disposed of valid accounts for drinking from a can actually took the can, only 64% of the participants without compelling justifications or excuses did so. When asked to indicate the "true" research question, none of the participants could specify the topic correctly.2 In addition to the technique of designing a plausible cover story, dependent variables of Type 3 measures are often assessed on an implicit or even physiological level. These measures are assumed to be immune against intentional distortion by the participant. In this textbook, implicit and physiological measures are discussed in separate chapters (see chapter 9 for the former and chapters 12 and 13 for the latter set of methods).
Type 2. In Type 2 measures, participants are informed about the general topic of an investigation, but particular hypotheses are hidden. These kinds of techniques might underlie the measurement most often used in psychology. Examples can be found in most questionnaire techniques measuring state or trait personality, situated cognitions, or emotions. Even though people most often know
2 A more thorough discussion of ethical considerations concerning deception studies is presented later in this chapter.
and are aware of the fact that they are actually being asked about their environmental attitudes, the degree of their introversion, or actual self-esteem in such studies, as a rule they do not know the specific hypotheses such as the item assignment to different scale dimensions or assumptions about relationships between variables or the expected results. However, in a narrower sense, not only the participants should be blind to specific hypotheses: To minimize possible experimenter effects beyond those associated with the mere research setting, assistants of the experimenter should not be informed about the hypotheses (Type 2) or even about the research question as a whole (Type 3). In a way, those research assistants who are involved in double-blind studies are de facto participants in the wider sense because they are (ideally) not the principal investigator. Being blind to particular hypotheses implies that it is not possible for participants to systematically counteract the primary research goal. Nevertheless, participants who know about the field of research they are contributing to might be motivated to extend their contribution to the researcher. They often generate their own hypotheses that they want to prove immediately by responding in a respective manner. Hence, Type 2 measurement might be flawed with increased unsystematic measurement error.
Type 1. In some studies it cannot be avoided that the participants are aware of a researcher's hypotheses. Yet there are also investigations where the hypotheses are deliberately disclosed to the participants. This might be done for ethical as well as for feasibility reasons. Even though revealing hypotheses is generally assumed to open the door to systematic distortion, this tendency can be counteracted by either decreasing or preventing altogether the participants' awareness of a measure's manipulability. This is the goal of Type 1 nonreactive measures. Although different strategies can be incorporated that might reduce the perceived manipulability (e.g., simply informing participants about a measure's nonmanipulability, using highly complex materials), the most prominent example of a Type 1 measure is the bogus pipeline technique (Jones &
Sigall, 1971). In studies using this technique, participants are led to believe that it is possible to pump their psyche directly using an apparatus that apparently records physiological signals. This technique has been found to considerably reduce reactivity in attitude assessment that is rooted in the social desirability concerns of the participants (for a review, see Roese & Jamieson, 1993).
Type 0. Until now we have expounded on the different types of measurement techniques that approximate nonreactivity to different degrees. To present a complete picture, Table 14.1 also includes Type 0 techniques, that is, measures that definitely do not fulfill the criterion of nonreactivity. In Type 0 measures, the interaction of researcher and participants is designed and perceived as serving the investigation of particular hypotheses, and the participant is fully aware of being able to manipulate the results. One might assume that an atmosphere of cooperation between researcher and participant, fueled by the full disclosure of all research hypotheses, could minimize a participant's possible tendency to sabotage the results or might even motivate participants to give their best. For some purposes this effect might have some benefits, for example, when the research is very dependent on the information provided by single participants as, for example, in the case of witnesses recounting rare events. However, the negative implications of fully informed participants for measurement quality prevail over the possible benefits. Research on the demand characteristics of psychological studies (e.g., Orne, 1962) has often described the tendency of cooperatively motivated subjects to distort their behavior or statements toward the assumed hypothesis. This fundamental danger (as well as the further pitfalls of reactivity) can be found in all forms of collaborative and consensual research techniques (for a discussion see, Page, 2000).
In the following sections, we will describe and discuss important nonreactive measures in more detail. After reviewing a few classical measures, a description will be given of the recent trends and techniques in nonreactive measurement that have in part developed because of sociotechnological changes of human behavior.
Was this article helpful?