Mode of Data Collection Paper Versus Electronic

Early versions of the ESM used pagers to signal participants to make a diary recording. Later versions used digital watches with auditory alarms and the capability of storing many (e.g., 100) preprogrammed dates and times for alarms. Some of the watches were linkable to personal computers so that stored alarm schedules could be easily downloaded.

Both the pager and watch methods have inherent limitations. First, because alarm schedules are preprogrammed based on "usual" awakening and bedtimes, participants who alter their wake-sleep schedules or who naturally have unusual schedules may be beeped when they are asleep and not beeped when awake. This can result in chunks of the participant's waking day not being included in the sampling scheme, which is a threat to the validity of the method. Some investigators have individualized sampling protocols by obtaining individuals' typical waking hours and have scheduled the sampling so that it falls within those hours. This helps resolve the validity threat, but does not allow for variability in daily wakening hours (e.g., weekday vs. weekend). Second, preprogrammed devices are limited in their flexibility. They do not have the capacity to alter the manner in which questions are presented to participants (context-specific shifts like program branching when a response indicates, e.g., that some questions are not relevant). They also do not have the capability to alter their sampling routine in accordance with the participants' behavior (e.g., alter the time to the next random beep based on the occurrence of event-driven beeps to prevent beeps from being too close to one another).

Technological developments in palmtop computers in the 1980s culminated in the current crop of Personal Digital Assistants (PDAs), which was a boon to the EMA held. These compact, fully functional computer devices allow the programming of sophisticated sampling schemes that can present questions to participants directly on the PDA screen (see Figure 5.2). Most of the screens are touch-sensitive, so responses as well as the questions themselves can be presented. This eliminates the need for paper diaries and a signaling device and replaces them with a single, compact unit, which stores the electronic diary (ED) data until the information is uploaded to a personal computer. Of course, this also eliminates the process of transcribing paper questionnaire responses to electronic form, a time-consuming and error-producing process.

Researchers have used EDs in many ways, but only some of them realized the full capacity of these

FIGURE 5.2. An electronic diary displaying a visual analog scale.

devices. Some implementations of EDs have mimicked the method of uploading preprogrammed alarms described above with watches; however, this method does not solve the sampling exposure problem. Other programs have been developed that generate prompts in real time and that result in the ability to adapt to participants' schedules. Some of the most sophisticated programs randomly implement a sampling strategy according to a set of parameters. For instance, if a random schedule was desired, the investigator would input both the average and minimum intervals between signals and the program would prompt the individual using these parameters. Thus, an individual who was awake for a 20-hour period would obviously receive a greater number of prompts compared with one who had a shorter number of hours awake, but both people could provide information during the entire time period they were awake.

One of the most compelling reasons for using EDs concerns the issue of participant compliance with sampling protocols, or the proportion of prompts completed according to the protocol. Although low levels of noncompliance probably do not unduly impact the validity of data, very high levels of noncompliance are detrimental to data quality. Data collected with paper diaries do not provide investigators with direct information about compliance; instead, compliance is inferred by examining the times and dates that participants record on the paper diaries. Inferred compliance rates have actually been quite high, at least at the 90% level (Hufford, Stone, Shiffman, Schwartz, & Broderick, 2002). However, researchers have been concerned about this apparently high level of compliance for paper diaries for a couple of reasons. First, experiences with instrumented medication delivery devices, where an unobtrusive computer monitors the use of the device (e.g., a steroid inhaler or a pill dispenser), have shown poor levels of compliance with medication protocols (Simmons, Nides, Rand, Wise, & Tashkin, 2000; Straka, Fish, Benson, & Suh, 1997). Second, many anecdotes from investigators running clinical trials with paper diaries suggest that diaries are often hoarded and completed at one time, sometimes right before a visit to the research site (known as "parking lot" compliance).

A recent study (Stone, Shiffman, Schwartz, Broderick, & Hufford, 2002; Stone et al., 2003) examined compliance with a sampling protocol wherein chronic pain patients were to make diary entries about their pain thrice daily (10 a.m., 4 p.m., and 8 p.m.), over a 3-week period. The study used an instrumented binder that held paper diary sheets and recorded the openings and closings of the binder. By defining periods of time around each of the targeted recording hours (e.g., ± 15 m or ±45 m) and examining the times when the diary binder was actually open, we could determine actual compliance rates. In summary, compliance rates computed by examining subjects' self-reported entry times and dates were consistent with past reports (about 90%). However, when actual compliance was computed based on the openings, compliance dropped dramatically to 11% (30 m window) and 19% (90 m window). More detailed examination of individual records produced evidence of back-filling of paper diaries and, surprisingly, of completion of diaries in advance (forward-filling). Altering the protocol to include an auditory reminder alarm just before the targeted completion times increased compliance only by about 10% (Broderick, Schwartz, Shiffman, Hufford, & Stone, 2003). For us, the data suggest the importance of actually confirming protocol compliance, which EDs are easily able to do. (A comparison group using EDs was also incorporated into this study using the same protocol; EDs that included auditory signals and other compliance enhancing features yielded an actual compliance rate of 94%.)

Application. EMA data collection protocols (using both paper and electronic diaries) have been applied to a variety of situations, including (but not limited to) pain assessment, symptom reports in clinical trials, smoking and alcohol cessation or cravings, food consumption, mood and stress assessments, and psychiatric and physiological symptoms (Hufford & Shields, 2002; Stone et al., 1998). In fact, it has been reported that about 25% of Phase 2 to Phase 4 clinical trials use some type of diary to monitor patients' responses (Hufford et al., 2002). These studies have examined both healthy volunteers as well as various disease populations.

Furthermore, studies have not been limited to adults; adolescents and children have also participated in studies using EDs (Henker, Whalen, & Jamner, 2002; Walker & Sorrells, 2002).

As technology has advanced, both clinicians and researchers have increasingly implemented electronic diaries with EMA designs. Hufford and Shields (Hufford & Shields, 2002) identified 76 empirical articles that used electronic diaries published in peer-reviewed, English language journals from 1990 to 2001. Since then, an additional 24 have been published (identified by replicating Huf-ford's original searches).

The ease of use of the devices and depth of information obtained using the various sampling strategies explain the shift to studies utilizing EMA and EDs. In addition, the ability to address within-person hypotheses that are clinically relevant (Stone et al., 2003) make this ideal for the interdisciplinary research conducted in behavioral medicine.

0 0

Post a comment