Is Verbal Data Psychometrically Good Data

There might be many good reasons to use text data for psychological research. From a measurement perspective, one of the most important questions concerns the extent to which verbal data is psychometrically good data. Unfortunately, it is common for text analysis researchers, after developing a new method, to proceed to its application without establishing its psychometric properties. Thorough construct validation in the area of text analysis is yet rare. However, at least two notable exemptions to this rule deserve to be mentioned. A large body of research has established the validity of TAT-based motive measures. From this it has become clear that implicit motives (a) can be reliably assessed with the TAT (Lundy, 1988; Smith, 1992; Tuerlinckx et al., 2002; Winter & Stewart, 1977), (b) are distinct from self-reported motives and traits (King, 1995; Schultheiss & Brunstein, 2001), and (c) uniquely predict types of behavior (McClelland, Koestner, & Weinberger, 1989; Winter et al., 1998).

The basic psychometric properties are also comparatively well understood for word count-based measures. Across a series of studies, the words that people use in their spoken and written language have emerged as stable over time and across context (Gleser, Gottschalk, & Watkins, 1959; Mehl & Pennebaker, 2003; Pennebaker & King, 1999; Schnurr, Rosenberg, & Oxman, 1986). Also, spontaneous word choice shows reliable and theoretically meaningful associations with demographic variables (Groom et al., 2004; Pennebaker & Stone, 2003) and traditional personality measures (Pennebaker et al., 2003), but also predicts, for example, real-life health behaviors over and beyond the Big Five dimensions (Pennebaker & King, 1999).

To summarize, the question to what extent text analysis yields good data from a measurement perspective is important and needs to be answered for each method separately. So far, at least for TATassessed motives and word count-based measures, the existing evidence suggests good psychometric properties. Thorough construct validation that, for example, establishes aspects of convergent validity between different text analysis methods (e.g., emotion words across different programs) and between text analysis methods and other psychological methods (e.g., self-reported, observed, and linguistic measures of emotions) are needed.

Was this article helpful?

0 0

Post a comment