Bias and Base Rates

Panic Away

Panic Away Program

Get Instant Access

As elaborated in Reynolds and Ramsay's chapter in the present volume, bias occurs in the utilization of assessment information when examiners allow preconceived notions and previously held beliefs to influence how they view the implications of their data. Assessment bias may arise either inadvertently, from attitudes of which examiners are unaware, or consciously, as purposeful intent on their part. Whether inadvertent or intentional, assessment bias takes the form of expectations that affect the meaning assigned to a set of findings, and most of these expectations originate in turn from demographic beliefs, environmental impressions, and epidemiological notions.

As an example of demographic beliefs, an assessor who thinks that older people and males are generally likely to perform better as managers than younger people and females may advise hiring a 45-year-old man and not hiring a 30-year-old woman for a managerial position, even if their psychological assessment information would be seen by most examiners as comparable or even favoring the female candidate. Similarly, an assessor who harbors a conviction that blue-collar African Americans are generally less likely to respond to psychotherapy than white-collar Caucasians may discourage psychotherapy for the former and recommend it for the latter, even when looking at assessment information showing equivalent treatment accessibility.

Environmental impressions as a source of biased expectations refer to the setting in which assessors are conducting an evaluation. Psychologists working in an inpatient facility in which a large percentage of patients are psychotically disturbed come to expect most of they people they examine to be psychotic, at least on admission, and they may accordingly be inclined to infer psychosis from a set of assessment data that would not have led them to this conclusion had they obtained it in an outpatient clinic in which psychotic disturbance is rarely seen. Similarly, psychologists assessing prison inmates, among whom antisocial personality disorder is commonly found, may be more likely to expect and diagnose this disorder than they would if they were working with similar data in a university counseling center.

As for epidemiological notions, examiners may be consciously or inadvertently influenced in the conclusions they draw by how they view the nature and incidence of various conditions. Those who believe that borderline personality disorder is widespread are likely to diagnose this condition more frequently than those who think this diagnostic category lacks precision and is used too frequently. Those who believe that attention-deficit/hyperactivity disorder (ADHD) occurs mainly in boys, and adolescent anorexia mainly in girls, are relatively unlikely to diagnose ADHD in girls and anorexia in boys.

In all such instances of possible influence derived from demographic, environmental, and epidemiological expectations, the challenge for assessment psychologists is to recognize their personal biases and prevent them as much as possible from exerting inappropriate influence on the conclusions and recommendations they derive from their assessment data. On the other hand, the previous examples were chosen to indicate that epidemiological and environmental expectations may have some basis in fact. There are more psychotic patients in hospital than in clinic populations, there are more antisocial individuals in prison than on college campuses, and there are substantial gender differences in the incidence of ADHD and anorexia. From a strictly actuarial point of view, then, being hospitalized does increase the probability of being psychotic, being incarcerated does increase the probability of being antisocial, and being male or female does increase the probability of being attention disordered or anorexic, respectively. Taking adequate account of such actual setting and group differences, while preventing them from resulting in biased conclusions, involves being alert to whatever base-rate information may be available in the individual case.

Base-rate information refers to the expected frequency of a characteristic or behavior in particular persons or circumstances. Attention to applicable base rates provides a way of estimating the utility of assessment procedures, particularly with respect to their efficiency in assessing rare events. As first identified by Meehl and Rosen (1955), base rates can become problematic for measuring instruments when the expected frequency of an event falls very far below 50%. For example, in a clinical setting in which 10% of the patients are suicidal, a valid test of suicidality that has a hit rate of 60% (i.e., is correct 60% of the time in identifying people in general as suicidal or nonsuicidal) is technically less efficient than simply calling all of the patients nonsuicidal, which would be correct 90% of the time.

Although technically correct from a psychometric perspective, this type of base-rate emphasis on efficiency does not always satisfy priorities in actual assessment practice. Assessment methods that are inefficient in assessing suicidal-ity, given its low base rate even in most patient populations, may nevertheless correctly identify a subgroup of patients in whom suicidal behavior is relatively likely to occur. An examiner can then use this information to recommend suicide precautions for this apparently suicidal subgroup, which is preferable to overlooking the self-destructive potential of the high-risk group by exercising the technically more efficient option of calling all of the patients nonsuicidal.

The base-rate problem can also be minimized by focusing assessment efforts on restricted populations in which the expected frequency of the characteristic being evaluated is less rare than in the general population. Kamphuis and Finn (2002) note in this regard that the more closely a base rate approximates 50%, the better prospects a valid measure has of improving on the efficiency of concluding that either everyone or no one has a certain characteristic or behavioral tendency. As an example of increasing the base rate by restricting the population, efficient prediction of violent behavior among people in general is difficult to achieve, because most people are nonviolent. In a population of criminal offenders, however, many of whom have a history of violence, a valid measure of violence potential may prove quite efficient in identifying those at greatest risk for violent behavior in the future.

Value Judgments and Cutting Scores

Value judgments in the present context refers to the purposes for which a respondent is being evaluated in relation to the frequency of false-positive and false-negative outcomes that an assessment variable is likely to produce. False-positive outcomes result in decisions based on assuming that people have certain conditions and tendencies that they in fact do not, whereas false-negative outcomes result in inferring that people lack certain conditions and tendencies that in actuality do characterize them. When assessments are being conducted to assist in making decisions about psychological characteristics and their consequences that most people would regard as undesirable, like being suicidal or homicidal, false positives may be of less concern than false negatives. A false-positive decision concerning dangerousness might result in a person's being unnecessarily supervised or even restrained, which is a regrettable but not a fatal outcome. A false-negative decision, on the other hand, by failing to identify dangerousness to oneself or others, can result in loss of life.

Conversely, false-positive outcomes may be more problematic than false-negative outcomes when referral questions concern desirable characteristics and consequences, like whether a person should be given a scholarship, a job, a promotion, or a parole. False negatives in this kind of assessment situation may result in denying people opportunities for which they are qualified and deserving, which is disadvantageous and perhaps unfair to them as individuals. However, when false positives result in promotion of personnel to positions of responsibility that exceed their capacities, or the parole of felons whose criminal tendencies have not abated, then many people other than the individual are likely to suffer serious consequences.

In relation to such value judgments, then, a set of assessment data may have different implications in difference assessment circumstances and thereby call for assessors to select carefully the cutting scores they utilize in formulating their conclusions and recommendations. For quantifiable dimensions of assessment that correlate positively with the presence of a characteristic or behavioral tendency, moving up the numerical scale produces a progressively decreasing percentage of false positives, and moving down the scale produces a progressively decreasing percentage of false negatives; just the opposite will be the case for assessment dimensions that are inversely correlated with what they measure. As a way of deciding the implications of assessment findings in a particular circumstance, cutting scores can thus be selected to minimize the likelihood of false-positive outcomes in examinations concerned with desirable consequences and minimize false-negative outcomes in the estimation of undesirable consequences.

Culture and Context

Just as assessment information may have different implications for people examined in different settings and for different purposes, it may also vary in its significance for respondents coming from different cultures or living in different social contexts. Hence the utilization phase of the assessment process must always take account of how characteristics of individuals identified in the interpretive phase are likely to affect their psychological functioning in their particular circumstances. Attention to cross-cultural influences has a long history in assessment psychology (see, e.g., Hallowell, 1951; Lindzey, 1961) and has seen a recent resurgence of interest, as described in the chapter by Geisinger in this volume and in contributions by Dana (1993, 2000b), Kazarian and Evans (1998), Suzuki, Ponterotto, and Meller (2000), and Williams, Satterwhite, and Saiz (1998).

The distinction drawn in this overview of the assessment process between interpreting and utilizing assessment information provides some useful guidelines for a two-step process in taking account of background and situational differences among respondents. The interpretive phase of assessment provides the first step, which consists of arriving at descriptive statements that identify a respondent's psychological characteristics as they exist independently of his or her cultural context and circumstances. Having superior intelligence, being orderly and compulsive, experiencing memory loss, being emotionally reserved, having an assertive and competitive bent, and being prone to acute anxiety in unfamiliar surroundings are examples of characteristics that define the nature of the individual. As revealed by assessment data, such characteristics will be present in people regardless of where they live, from whence they come, and in what they are involved. The utilization phase of the assessment process provides the second step, which involves being sufficiently sensitive to respondents' cultural and experiential contexts to estimate accurately the implications of their psychological characteristics in their particular life circumstances. Especially important in this regard is determining whether their psychological characteristics are likely to prove adaptive or maladaptive in their everyday world and what kinds of successful or unsuccessful adaptation might result from these characteristics in their particular circumstances.

Research findings document that cultural differences can lead to cross-cultural variation in modal psychological characteristics, and that the demands and expectations people face often determine the implications and consequences of particular characteristics, especially with respect to how adaptive they are (see Kazarian & Evans, 1998). For example, a generally passive, dependent, agreeable, and acquiescent person may be prone to adjustment difficulties in a cultural context that values autonomy, self-reliance, assertiveness, and competitiveness. Conversely, a fiercely independent and highly competitive person might feel comfortable and flourish psychologically in a subculture that values assertiveness, but might feel alienated and adapt poorly in a society that subordinates individual needs and preferences to the wishes and welfare of the group, and in which a passive and acquiescent person would get along very well.

These contextual influences on the implications of psychological characteristics extend to specific circumstances in persons' lives as well as their broader sociocultural contexts. A modest level of intelligence can be a source of comfort and success to young people whose personal and family expectations are simply that they graduate from high school, but a source of failure and dismay to those for whom graduation from a prestigious college is a minimum expectation. Similarly, a person with good coping skills and abundant adaptive capacities who is carrying a heavy burden of responsibilities and confronting numerous obstacles to meeting them may be susceptible to anxiety, irritability, and other consequences of a stress overload, whereas a person with limited coping skills and few adaptive capacities who is leading a narrowly restricted life involving very few demands may be able to maintain a comfortable psychological equilibrium and experience little in the way of subjectively felt distress. Likewise, a contemplative person who values being as careful as possible in completing tasks and arriving at conclusions may perform well in a job situation that calls for accuracy and thoroughness and involves relatively little time pressure, but may perform poorly in a position involving strict deadlines or requiring quick decisions on the basis of sketchy information, and in which a more decisive and action-oriented person would function more effectively.

As illustrated by the final example and those that have preceded it in this chapter, psychological assessment is a complex process. Diverse perspectives and attention to interacting variables are necessary in assessment psychology as elsewhere in behavioral science to expand knowledge and guide its practical application, and there is little to be gained from doctrinaire pronouncements of unidimensional approaches. To collect, interpret, and utilize assessment data effectively, one must give each of the issues identified in this introduction to the assessment process its just due, and the 24 chapters that follow are designed for this purpose.

Was this article helpful?

0 0
ADHD Secrets Uncovered

ADHD Secrets Uncovered

Announcing an important message for parents. Has Your Child Been Diagnosed With ADHD Is Coping With Your Child's Behavior Wearing You Out Are You Tired of Searching For Answers

Get My Free Ebook


Post a comment