Experimental Methods In Survey Research

All the methods discussed previously largely assume that truth is known to the respondents and that it is a dichotomous variable. There are, however, situations in which there is no absolute truth and in which the "real" answer may not be an unequivocal true or false. For example, a common assumption in cognitively oriented survey research is that people construct judgments of attitudes or behavior on the spot on demand, using the information that is accessible from their memories at the time of judgment (e.g., Schwarz & Bless, 1992; Strack & Martin, 1987). Some of this information may be chronically accessible and brought to mind whenever an issue is being referred to, resulting in judgments that are relatively stable over time. Other information, however, may be less accessible, and the context is an important determinant of whether it will be activated. Such information may inadvertently be made accessible to respondents via features of a self-report instrument as, for example, the presentation order of questions or the range of numerical response scales. This temporarily accessible information is capable of influencing the respondent during any of the stages of the response process, including question interpretation, retrieval of information from memory, judgment, and the generation of a response. In the worst case, the effects of such temporarily accessible information may lead to systematic measurement error. Experi mental methods of controlling the effects of contextual features on response processes can help to address this problem.

For example, according to the inclusion-exclusion model (Schwarz & Bless, 1992), assimilation occurs when information retrieved for answering a preceding question comes to mind and is used to form a temporary representation of the target of the current question. On the other hand, contrast effects occur when the information retrieved for an earlier question is excluded from the respondent's temporary representation of the target. The experimental variation of question order makes such context effects visible and also provides a means of controlling them.

Along a similar vein, several studies have shown that the range of frequencies in response options also affects the response process of survey respondents (e.g., Schwarz, Hippler, Deutsch, & Strack, 1985). For example, in an experimental investigation of response option ranges, respondents who were presented with a high-frequency range of responses (from "4 or less" to "9 or more") indicated to have been more often emotionally depressed during the past month than respondents presented with a lower range of response options (from "0" to "5 or more"; Harrison & McLaughlin, 1996). Again, only an experimental variation of different response options allows detecting and addressing such effects. Additional variables affecting the likelihood and direction of item order effects, as well as ways of addressing them, are summarized in Schwarz (1999); Sudman, Bradburn, and Schwarz (1996); Tourangeau (1999); and Tourangeau, Singer, and Presser (2003).

Was this article helpful?

0 0

Post a comment