Experimental Assessment Of Truth

Survey research has proved to be extremely useful for measuring opinions, attitudes, and behaviors across a broad spectrum of interest, including the most highly sensitive and controversial topics. But can a researcher really expect to get honest answers when asking for sensitive, socially disapproved, or incriminating attitudes and behaviors? In this section, we discuss four instructive examples of experimental assessment methods, all of which are concerned with the assessment of truth in sensitive areas. The first two of these methods, the randomized response technique (RRT) and the unmatched count technique (UCT), are examples for assessments at the group level, whereas the control question technique (CQT) and the guilty knowledge test (GKT) provide assessments at the level of single individuals. The RRT and the UCT are used in surveys to ask sensitive questions respondents might be willing to answer in principle, given that they are assured complete anonymity in a credible manner. In a more coercive way, the CQT and the GKT are used in polygraph tests to uncover the truth in cases in which guilty respondents are by no means willing to tell the truth because they have to fear serious consequences on disclosure of their misdoings.

The randomized response technique (RRT) was first suggested by Warner (1965). He based his technique on the notion that arguably the most promising method to encourage honest responding in surveys is to collect data anonymously To credibly ensure respondents' anonymity, Warner directed the respondents to answer to one of two logical opposites, depending on the outcome of a randomizing device that selects the question to be answered with probability p and (1 - p), respectively. For example, a randomized response survey may consist of the following set of questions pertaining to a stigmatized Group A (which may consist, e.g., of tax evaders or marijuana consumers):

Question 1: Is it true that you are a member of A? Question 2: Is it true that you are not a member of A?

The respondent is asked to answer "yes" or "no" to one of these questions. Importantly, a randomizing device (e.g., dice) is used to determine the question to which the respondent is asked to answer. With probability p, the respondent is asked to answer Question 1; with probability (1 - p), the respondent is asked to answer Question 2. Even though the researcher does not need nor want to know the outcome of the dice throw and, consequently, the question that is actually being answered, he does know the probability p that is determined by the nature of the randomizing device and can, therefore, use elementary probability theory to determine the percentage of respondents' affirmative responses at the aggregate level:

where K denotes the proportion of the total population belonging to the stigmatized group A. Equation (5) defines the measurement model of the RRT; it relates the to-be-measured latent parameter n to the overall proportion of "yes" responses, P(yes), that can be estimated directly from the data. Note that because the outcome of the randomizing device is unknown to the interviewer, the respondent's anonymity is guaranteed; nobody can know whether a given "yes" answer indicates that a respondent belongs to the sensitive group. However, solving for n in the preceding model equation, an estimate of the proportion of respondents being a member of the stigmatized Group A can be obtained from the proportion of "yes" responses in the sample and p, the probability determined by the randomization device (Warner, 1965):

Warner's original formulation of the RRT was followed by many improvements aimed at enhancing the validity of the approach. For example, to address the problem that the efficiency of the model is less than optimal, the sensitive question may be paired with an unrelated question. The following questions may then be presented to the two groups to which each respondent is assigned with probability p and 1 - p, respectively:

Question 1: Have you ever used heroin? Question 2: Do you subscribe to Newsweek1

The prevalence of "yes" responses to the neutral question has to be known in the randomized response estimation procedure. If it is not known a priori, additional empirical evidence is required. Alternatively, the directed-answer variant of the RRT may be used in which the respondents are either asked to report truthfully to the sensitive question or to ignore the question altogether and to just say "yes," thereby also protecting those who give a "yes" answer to the sensitive question (Fox & Tracy, 1986; Greenberg, Abdul-Ela, Simmons, & Horvitz, 1969). In each case, the prevalence of the sensitive attribute in the population may be estimated from the individual responses, while the anonymity of each individual respondent is upheld.

The feasibility of the RRT has been demonstrated in a large number of studies. The questions at issue concerned drug abuse, exam cheating, illegal abortions, Social Security fraud, child abuse, tax evasion, and a host of other sensitive topics. In most validation studies, the randomized response approach produced considerably higher estimates of K than did the "yes" responses of direct questioning, thus providing evidence for the usefulness and validity of the approach. Detailed summaries of this research can be found in Antonak and Livneh (1995), Chaudhuri and Mukerjee (1988), Fox and Tracy (1986), and Scheers (1992).

For more than 30 years, all randomized response models tried to divide the population into two distinct and exhaustive classes: those respondents who engaged in the critical behavior and those respondents who did not. The respective sizes of these classes were represented by the population parameters p and b, respectively. Because these two parameters add up to 1, only one parameter had to be estimated. This could easily be done based on the one data category available in most RRT models, namely, the overall proportion of "yes" responses. However, despite their many successful applications, traditional RRT approaches can be criticized as being susceptible to cheaters, that is, respondents who do not answer as directed by the randomizing device. There is indeed evidence that cheating does occur (Locander, Sudman, & Bradburn, 1976). Clark and Desharnais (1998) therefore proposed an extension to the traditional RRT technique that no longer assumes that all respondents necessarily conform to the rules of the RRT. In their cheater detection model, Clark and Desharnais (1998) took into account that some respondents may answer "no," regardless of the outcome of the randomizing device. Their model, therefore, endeavors to divide the population into the following three classes: p (the proportion of compliant and honest "yes" respondents, i.e., respondents who honestly admit the critical behavior); b (the proportion of compliant and honest "no" respondents, i.e., respondents who truthfully deny the critical behavior); and g (1 -p-b, the proportion of noncompliant respondents who do not conform to the rules of the RRT and answer "no" to the sensitive question, regardless of the outcome of the randomization process). Obviously, there are two independent parameters in this model (because the three proportions add up to 1), and two parameters cannot be estimated on the basis of only one proportion of "yes" responses provided by traditional RRT methods. The problem of parameter estimation for this model can, however, be solved by an experimental between-subject manipulation of the probability with which participants are forced by the randomizing device to simply say "yes." Thus, the experimental approach to psychological assessment again leads to a considerable improvement in the assessment quality. By assigning participants randomly to two groups for which the probability of being forced to say "yes" by the randomizing device is different, the null hypothesis that no cheating occurs (g = 0) can be tested.

Another interesting and appealing alternative to the traditional randomized response method was developed by Miller (1984). In what has been called the unmatched count or randomized list technique (RLT/UCT), much of the complexity and distrust sometimes associated with the use of randomizing devices and the seemingly bizarre instructions used in the RRT are avoided. In the UCT, respondents are simply given a list of behaviors including the sensitive behavior of interest as well as a number of innocuous additional items. The respondent is then asked to report in total how many of the activities in the list he or she has engaged in. The assumption is that the respondent will feel comfortable reporting this total count because it does not reveal any particular activities he or she has been involved in. In an experimental between-subjects manipulation, a second sample of respondents is given a similar list that, however, does not contain the sensitive question. Let ¡J. denote the mean number of activities people engage in without the critical activity. If K again represents the proportion of the total population engaging in the critical activity, the model equations of the UCT describing the mean counts (Xj and |i2 in experimental Conditions 1 and 2, respectively, as a function of |i and n can be written as follows:

Thus, by subtracting the mean counts in the two samples, an estimate of n, the prevalence of the sensitive behavior, may be obtained as illustrated in Table 15.1.

Despite its apparent simplicity, some caution must be exercised when using this technique. For example, if the nonsensitive items are uncommon, any total count greater than zero will rouse suspicion. On the other hand, if the nonsensitive items are common, the total count can reach its theoretical maximum, no longer offering protection to the respondents. Fox and Tracy (1986) therefore recommended the use of (a) as many items as feasible and (b) items ranging midway between a 0 and 100% prevalence, making extreme total counts unlikely. Applied appropriately, the randomized list technique has been shown to lead to higher estimates for sensitive behaviors than could be obtained using direct questioning (LaBrie & Earleywine, 2000; Wimbush & Dalton, 1997).

What is common to both the randomized response and the unmatched count technique is that because of their between-subjects design, they can only be used to determine the prevalence of the target behavior at an aggregate level. The status of a single individual can never be determined without undermining the honest promise of anonymity on which these methods are based. However, there are situations—most often in the course of police investigations and legal proceedings—in which the

Was this article helpful?

0 0
Drug Free Life

Drug Free Life

How To Beat Drugs And Be On Your Way To Full Recovery. In this book, you will learn all about: Background Info On Drugs, Psychological Treatments Statistics, Rehab, Hypnosis and Much MORE.

Get My Free Ebook

Post a comment