Some Alternative Metrics

In an ideal world of construct validity, we would collect data whose assessment methods share a minimum of facets (e.g., structured vs. semistruc-

tured vs. unstructured, paper-and-pencil vs. observations by others, verbal vs. behavioral, accessible vs. inferred by observers, transparent vs. masked) with other assessments of these constructs. As this book details, other areas of psychology have developed metrics that could and should be applied to I/O research. These other assessments could include such measures as behavioral traces (Fritsche & Linneweber, chap. 14, this volume; Webb, Campbell, Schwartz, & Sechrest, 1966), direct observational reports (Neyer, chap. 4, this volume), semistructured techniques (Hulin & Maher, 1959), peer reports, self-reports that minimize verbal content (Kunin, 1955), event or signal sampling (Stone & Litcher-Kelly, chap. 5, this volume), response times to attitude items (Fazio & Olson, 2003; Fazio & Williams, 1986; Robinson & Neighbors, chap. 9, this volume), and objective metrics. Explorations of theory/data interfaces using alternative research disciplines such as computational modeling (Glomb & Miner, 2002; Ilgen & Hulin, 2000; Seitz, Hulin, & Hanisch, 2000; Seitz & Miner, 2002) should also be included in our research techniques.

One area we would like to highlight is reaction time measures. Such measures allow inferences about cognitive processes that are inaccessible with verbal self-reports. Use of such metrics opened up new areas of conceptual development about many constructs and popularized new areas of study. For example, the use of reaction time metrics permitted scientists to study the organization of material in memory (Schneider & Shiffrin, 1985) and the categorization of objects in perception and memory (Benjamin, chap. 24, this volume). In social psychology, such measurement operations have permitted theory to extend to such areas as "implicit attitudes," or attitudes that are not readily verbally accessible and reportable or reportable (Greenwald & Banaji, 1995; Greenwald, McGhee, & Schwartz, 1998), or attitude accessibility (Fazio & Williams, 1986).

Research indicates that chronically accessible attitudes are better predictors of some behaviors than are attitudes less chronically accessible. For example, Fazio and Williams (1986) showed that attitude accessibility was a predictor of behavior after controlling for attitude strength in a study of voting intentions in the months preceding the 1984 presidential election. Individuals with chronically accessible attitudes toward the candidates voted how they said they would more often than did those with less-accessible attitudes, even though both groups rated the candidates identically.

Even though many of our most popular constructs are attitudes (e.g., job satisfaction, organizational commitment, justice), little work has been done on accessibility within the field of I/O. When a participant in a study answers "Yes" to the query, "Are your coworkers boring?" we do not know whether they constantly evaluate their coworkers as boring and avoid them as much as possible, or whether they just escaped from a long and boring conversation with colleagues. Of course, on average, we'd expect people with more negative overall evaluations to respond affirmatively more often. However, the modest relations between attitudes and relevant behaviors suggest attention paid to accessibility of attitudes might pay substantial dividends in the strength and generality of relations between job attitudes and behaviors. It is also possible that less-accessible attitudes represent an independent pool of variance that is reliably related to different outcomes than chronically accessible attitudes.

The issue of chronic accessibility is broader than responses to structured questionnaire items. It deals with a characteristic of our theories. We assume the constructs we measure influence outcomes consistently across time. In the case of weak, or nonchronically accessible, or attitudes whose accessibility varies depending on affective factors, this may not be a safe assumption. Worse, it is unlikely that this assumption causes only random error in prediction. If only the subset of people with chronically accessible attitudes drives our results, we may be generalizing inappropriately.

Chronic accessibility might matter little for I/O psychology. The case could be made (e.g., Hulin, 2001) that work represents such an important part of most people's lives, that attitudes about work will always be chronically accessible. This may be why job attitude/job behavior relations are typically stronger than more elusive social attitude/social behavior relations (Hulin & Judge, 2003); it should be investigated. The point is that without using reaction time measures or alternate metrics to complement verbal self-reports, I/O researchers will fail to investigate this potentially important component of an immensely popular topic within the held.

Was this article helpful?

0 0

Post a comment