Recall Biases In Selfreport Data

A major theme in the development of diary and momentary capture of self-report data is the possibility that biases contaminate recall. As it turns out, at least three fields of scientific study have contributed to our knowledge of these biases: survey research, autobiographical memory research, and cognitive science. Many excellent reviews of factors exist that can influence self-reports (e.g., Bradburn, Rips, & Shevell, 1987; Gorin & Stone, 2001; Schwarz, Wanke, & Bless, 1994; Schwarz, 1999), and we describe several of the major factors below.

Figure 5.1 shows a graphical depiction of biasing factors. The figure shows how these factors influence

FIGURE 5.1. Schematic of factors that can influence a recall judgment.

Person Factors

Neuroticism Depression Optimism Private self-consciousness

Motivation to dissemble

FIGURE 5.1. Schematic of factors that can influence a recall judgment.

the recall rating, which is represented by the circle labeled "Judgment." We have chosen this term because the process of recalling information is best viewed as making a judgment, implying a dynamic process comprised of the differential weighting of information. To the left of the circle is a jagged line indicating the level of the variable to be recalled (for ease of presentation, we will use pain) studied over the period of time (e.g., a week). It is depicted as having considerable variation around its average level (the dotted line). Each of the rectangles signifies a potential recall biasing effect. The box labeled "Recency" means that making a judgment of pain for the entire time period is overly influenced by recent pain levels: if recent pain was high, the judgment of weekly pain would be higher. Similarly, the box labeled "Peak" indicates that peak experiences, in this case a pain exacerbation, will also overly influence the judgment of weekly pain. Both peak and recency effects have been elegantly demonstrated by Redelmeier and Kahneman (1996; Kahneman, Fredrickson, Schreiber, & Redelmeier, 1993).

"Effort" is shorthand for the term effort-after-meaning, which refers to a process wherein the recall of experiences that occur early on in the period is influenced by events occurring later in the period (Brown & Harris, 1978). An argument with one's boss, for instance, might be deemed very upsetting at the time. However, if reconciliation with the boss occurs a day later, then subsequent recall of the original level of upset could be affected by the reconciliation, diminishing the recalled stressfulness of the event. In other words, the memory of past events can be colored by later events. "Summary" refers to the process of taking whatever information one has about the experience from a particular period and creating a single rating from this information. In fact, we know that only a small subset of all experiences is encoded into memory, which means that all information is not even available for summarizing. We also know that the process of retrieving this stored information is an active process, influenced by a number of factors. Finally, it is unclear how people mathematically combine the retrieved information into a meaningful index. This is an area in need of research.

The box labeled "Immediate Context" is a very influential and well-researched factor. We know that a number of cognitive heuristics come into play during the moment of judgment. The degree to which the process is influenced is itself affected by the nature of the information to be recalled (Menon & Yorkston, 2000); one quality of the information especially salient at this stage is the degree to which the individual has access to the information requested by the researcher. When asked for a rating of how sad one is feeling, for instance, a person usually has an immediate response stemming from their current experience. However, when asked about how their life is currently going (life satisfaction), there is not an immediate experience, but rather a process of evaluation that occurs. In the latter case, when the answer to the query is not apparent, immediate circumstances (e.g., the person's mood) have a greater influence on the response. So, a person in a happy mood is more likely to say that they are satisfied with life than a person who is currently sad, even if they actually have quite similar lives. Schwarz and Menon, among others, have detailed the cognitions underlying many of these processes (Menon & Yorkston, 2000; Schwarz, 1999). Surprisingly, even seemingly trivial manipulations of circumstances surrounding judgments can have a major impact on the judgments.

The large box labeled "Person Factors" on the right side of Figure 5.1 shows a number of personality traits and concepts shown to influence judgments of past experiences. Several of these variables, among many, have been listed.

Other research on the types of memory that are used in the recall process have provided an additional line of evidence on recall bias. Episodic knowledge has been described by cognitive scientists as representing the autobiographical experiences that are linked to specific events; its form is loosely represented in memory, and it is susceptible to forgetting. Semantic memory may be considered a symbolic form of memory that is conceptual in form; it is more tightly linked to individuals' beliefs about the world than to specific occurrences. In recent work on the remembrance of emotions, Robinson and Clore (2002a, 2002b) have demonstrated evidence of a shift from episodic memory to semantic memory as the interval to be remembered is increased (in particular, latencies to recall information increased and then decreased as the interval for recall increased). This may show a tendency to shift from summarizing specific experiences to reporting beliefs about what those experiences are generally like, when people are asked to recall information over long periods. We expect that advances in the cognitive and brain sciences will inform our understanding of recall bias.


To minimize the potential for recall bias and understand people within the context of their normal environment, more intense momentary data collection protocols were developed. In particular, Experience Sampling Method (ESM; Csikszenmihalyi & Larson, 1987; DeVries, 1992) sprang from the early studies of Csikszentmihalyi on the "flow," which intensely examined individuals in order to understand the interactions between experiences during their daily life (Csikszentmihalyi, 1994). The methodological advance (over the use of end-of-day diaries) was the collection of information about how the individual was feeling at the moment preceding an electronically administered auditory signal. The technology used in these studies involved having study participants carry electronic pagers activated by a central station; the investigators provided the central station with a list of times throughout the day when pagers should be activated. When signaled, individuals were to record information on a pocket-size diary, where one or two pages of the multipage diary would be used for each beep. A typical study might be comprised of 7 "beeps" per day for a 1-week period.

0 0

Post a comment