Methods Are Hypothetical Constructs

The language in which methods are often treated in the literature suggests that they are something technical, nonpsychological, or different than substantive variables. But method is a summary concept for a multitude of ways in which we obtain psychological information. The result of a method has psychological significance as does the method itself. Returning to the neighbor example, self-report, other-report, and observation were introduced as methods because the different procedures collected...

Latent Variable Models

The correlation and association methods described so far are correlations between observed variables that are usually affected by measurement error. Latent variable models are statistical approaches designed to separate measurement error from true individual differences. Moreover, latent variable models allow the definition of latent variables that represent different sources of influence on the observed variables. The advantage here is that one can model complex structures that link latent...

Prerequisite Psychometric Conditions

The use of CFA to model a multitrait by multisource matrix requires a significant amount of psychometric work on the rating scale for each source prior to this analysis. First, the items and features of the scale (e.g., wording of items, rating interval, rating anchors, and so on) must have good content validity (i.e., the representativeness and relevance of the items for the given construct Haynes, Richard, & Kubany, 1995). Second, the distributional characteristics of measures from each...

Guidelines For Using Experimental Assessment Methods

The previous sections of this chapter have shown how experimental methods can help address problems of psychological assessment for which alternative solutions based on more traditional methods still appear to be lacking. Therefore, it should not come as a surprise that experimental assessment methods have become more and more frequent in psychological research during the past two decades. In the area of memory assessment, for example, both the process dissociation procedure of measuring...

Formulating a Response

Once respondents understand a self-report question, they must formulate an answer. The processes involved in constructing an answer vary depending on the type of self-report being made. When reporting on the frequency of a specific behavior, for instance, respondents might be able to search their memory, count the number of occasions on which the behavior occurred, and report the counted value. When reporting an attitude, on the other hand, respondents must search their memory for relevant...

Confirmatory Factor Analysis Approaches

When data are continuous and there are one or more underlying factors, confirmatory factor analysis procedures may be used to test measurement invariance. Meredith (1993) considered the issue of measurement invariance across groups, and he developed a sequence of model comparisons that provide a close parallel to the IRT approach. Widaman and Reise (1997) presented a clear description of these procedures, and Meredith and Horn (2001) have recently extended this approach to testing measurement...

Gottschalk Gleser Method of Content Analysis

Also in the 1960s, Gottschalk and his colleagues started developing what became known as the Gottschalk-Gleser Method of content analysis (Gottschalk, 1995). The Gottschalk-Gleser Method involves participants giving a 5-minute speech on a personal life experience. The verbatim transcripts are then submitted to a content analysis. Several scales tapping into what Gottschalk calls psychobiological dimensions have been developed and validated. Most of the scales are derived from a psychoanalytic...

Multimethod Item Response Theory

Item response theory (IRT) is a framework for an increasing number of statistical models that refer to the same kind of data structure. The data basis for applying IRT models is a matrix of responses of a (large) number of persons on a (small) number of questions, tasks, stimuli, or whatever, called the items. The item responses may be dichotomous (yes-no correct-incorrect true-false, etc.), ordinal (strongly disagree, disagree, agree, strongly agree), or nominal without a given order of the...

Info

Many analytic methods can be used with any of the techniques, depending on the design of the study and the given measure. Note. Many analytic methods can be used with any of the techniques, depending on the design of the study and the given measure. Researchers interested in the role of cognition in emotion-related regulation are likely to assess either dispositional or situational cognitive processes (e.g., how individuals appraise specific emotion-eliciting situations and use cognitive...

Self Report Emotion Mood and Coping

Self-report methods are commonly used in developmental psychology. However, several important issues should be noted when using self-report data. First, age is an obvious concern because most children under age 8 have difficulty completing paper-and-pencil measures. Nevertheless, as we discuss briefly in this section, methods have been constructed that appear to successfully elicit young children's reports of some constructs. Second, the validity of self-report data sometimes is a concern...

Using Multiple Methods to Assess Different Predictors of an Outcome Variable

Many classical studies in health psychology test the various health behavior models that have been proposed since the 1970s. The most prominent models include structural models such as the health belief model (Becker, 1974 Rosenstock, 1974), the theory of reasoned action planned behavior (Ajzen & Fish-bein, 1980 Fishbein & Ajzen, 1975), and the protection motivation theory (Rogers, 1983), as well as more dynamic health behavior models that describe the processes of health behavior change...

Applications

We will illustrate the five models by applying them to data from an MTMM study exploring the relations between self- and peer-rated frequency of negative emotions. The traits were ear, anger, and sadness. The three methods were self-ratings, ratings by a good friend, and ratings by an acquaintance. The sample consisted of 172 triples of self- and peer raters. This sample was a subgroup of individuals from a larger MTMM study (see Eid et al., 2003 Trierweiler, Eid, & Lischetzke, 2002). While...

Proportion Agreement Index Percentage Agreement Index

The proportion agreement index (p0), which indicates how often two observers' ratings concur, is an intuitive and useful first measure of observer agreement. It is computed by dividing the number of times raters agree by the number of objects rated Jiy denotes the number of cases in the cell ij of the cross-classified table, and nu represents the cells on the main diagonal (where i j), which indicate concordant ratings. The same information is provided by the percentage agreement index (p ),...

Analysis Of Facet Data

Facet design is part of a more general approach called facet theory, which uses the facet structure to generate hypotheses about similarities between items. Facet theory relies almost exclusively on producing low-dimensional geometric representations of the data, which are then interpreted in terms of the properties of the defining facets (cf. Borg & Shye, 1995). Other approaches include confirmatory factor analysis (cf. Mellenbergh, Kelderman, Stijlen, & Zondag, 1979). A problem with...

Reporting a Response

The final step in making a self-reported judgment is to communicate that judgment to the investigator. Anything that impedes the accuracy of this communication will affect the validity of the report. For instance, when reporting on illegal or other socially undesirable behaviors, participants may simply decide not tell the truth. Alternatively, when asked to respond using a Likert response scale, respondents may attempt to provide an accurate response, but different respondents may use the...

Experimental Methods Of Psychological Assessment

What can be gained from applying methods of experimental psychology to problems of psychological assessment Experimental psychology and psychological assessment, although both being important branches of psychology, are clearly distinct scientific disciplines with unique histories, characterized by discipline-specific theories, paradigms, and research methods (see Bringmann, L ck, Miller, & Early, 1997). Consequently, there has been little overlap between research in experimental psychology...

Advantages And Disadvantages Of Selfreport Methods

Self-report methods offer clear advantages over other assessment techniques. These methods are simple, quick, inexpensive, flexible, and often provide information that would be difficult or impossible to obtain any other way Yet each advantage corresponds to specific disadvantages that may go unnoticed by researchers. For example, the ubiquity of self-report techniques results from the fact that they are so easy to administer. However, this ease of use may result in an overreliance on...

Integrating Neuroimaging Data With Other Data in a Multitrait Multimethod Framework

Assessment of psychological constructs has traditionally focused on behaviors that are either directly observable by a researcher or can be reported by the examinee. Neuroimaging can supplement these methods of assessment by providing information at a neural level. Although, one might be tempted to view this at a causal level (i.e., the brain activity causes the behavior, or the behavior causes the brain activity), it need not be viewed as such. Rather, neuroimaging data can be viewed as just...

The Weighted Average Model WAM

Whereas the SRM is a statistical model designed for the decomposition of the components of interpersonal perception, the WAM is a general theoretical model of perception sometimes applied to informant assessment to predict its qualities (e.g., consensus and self-other agreement). The WAM predicts that the qualities of informants' ratings are a weighted function of nine components 1. acquaintance (i.e., amount of information informants have about the target), 2. overlap (i.e., the number of...

Correlation and Association Models

The convergence of methods can be assessed by correlating the methods that are supposed to measure the same trait. Campbell and Fiske (1959) have extended this idea by defining an MTMM correlation matrix. In this matrix there is one indicator for each trait-method unit, and this matrix allows a thorough analysis of convergent and discriminant validity by comparing several correlation coefficients (e.g., the correlations between different methods measuring the same trait versus those between...

CFA Approach to MTMM Studies of Different Self Concept Instruments

What is the relation between scales based on responses from new and existing self-concept instruments Historically, self-concept research was plagued by a surfeit of idiosyncratic instruments that hindered communication among researchers and research syntheses. This situation invited the Jingle-Jangle Fallacy (Marsh, 1994) whereby researchers mistakenly assumed that two scales with the same label measured the same construct or that two scales with different labels measured different constructs....

Longitudinal Designs

Developmental psychology is concerned with intrain-dividual change over time. Therefore longitudinal methods (Khoo, West, Wu, & Kwok, chap. 21, this volume), which involve assessments of the same group of people over time, are well suited to the goals of developmental psychology. Because developmental psychologists are particularly interested in pathways and trajectories, designs that are able to capture the progression of time and the course of a phenomenon are especially valuable....

Representing Observational Data

With many measurement approaches, the question, How should one represent one's data does not arise. The standard rectangular data matrix suffices. Rows represent sampling units (participants, dyads, etc.), columns represent variables, and columns are filled in with the relatively few scores generated by the measurement approach. That is all the standard statistical packages need or expect, and even a preliminary step like scoring the items of a self-esteem scale, for example, is relatively...

The Experimental Method In Psychology

From the very beginning, the experimental method has been closely tied to hypothesis testing and theory evaluation in psychological research (see Boring, 1950 Bredenkamp, 2001 Calfee, 1985 Cook & Campbell, 1979 Davis, 1995 Shadish, Cook, & Campbell, 2002). During the past 50 to 60 years in particular, experiments have been used fairly routinely for testing hypotheses from different branches of psychology. Typical examples include frustration causes aggression (Berkowitz, 1989, p. 61),...

P0xaocLfO17

In this equation, L stands for the likelihood function, and the latent trait distribution is denoted by f(d). Warm (1989) used a prior distribution for J(6), which is equal to the square root of the Fisher information function. Such a prior is called nonin-jormative because it is a constant over the latent dimension and does not contain information about the latent distribution of person abilities. By means of Warm's method, an estimator for 6 is obtained that has a finite value for persons who...

Observational Methods Parent Child Relations

Observational methods are excellent tools for the study of the complex relationships examined in developmental psychology (Bakeman & Gnisci, chap. 10, this volume). One of the key distinctions between the approach of many developmentalists and that of some other disciplines is the importance placed on context. Individuals do not exist in isolation they actively construct their environment and simultaneously are influenced by their environment. The complexity of these interactions is often...

Validity

Validity, one of the key issues of research, concerns the question whether the inferences drawn from the results of a study are true or not (Shadish, Cook, & Campbell, 2002). In particular, with respect to measurement methods, validity represents the degree to which the adequacy and appropriateness of inferences and actions based on the results of a measurement device are supported by empirical evidence and theoretical rationales (Messick, 1989). Multimethod research plays a key role in the...

Reliability Of Observational Data

The standard psychometric concerns of reliability and validity are in no way unique to observational methods. The precision and accuracy of any measuring device needs to be established before weight can be given to the data collected with it. Such considerations apply no matter whether observational methods or other measuring approaches are used. Nonetheless, for some measuring approaches, reliability issues do not loom large. For example, usually we assume that, once calibrated,...

Franz J Ney er

Knowledgeable informants are frequently employed as data-gathering instruments in all domains of research in psychology. Informant assessments correspond with one of the basic data types known in psychology, which has been called L-data (i.e., life data recorded by observers) by Cattell (1957), 0-data (i.e., data generated by observers) by Block (1977), or l-data (i.e., data derived from informants) by Funder (2004). Informants are people who usually share some brief history with a studied...

CFA Approaches to MTMM Data

Using CFA approaches to MTMM data, researchers can define models that posit a priori trait and method factors and test the ability of such models to fit the data. In the general MTMM model (Marsh, 1989 Marsh & Grayson, 1995 Widaman, 1985) (a) there are at least three traits (T 3) and three methods (M 3) (b) T X M measured variables are used to infer T + Ma priori factors (c) each measured variable loads on one trait factor and one method factor but is constrained so as not to load on any...

Structural Equation Models For Multitraitmultimethod Data

Michael Eid, Tanja Lischetzke, and Fridtjof W Nussbeck Models of confirmatory factor analysis (CFA) or structural equation modeling (SEM) have generally become the most often applied methodological approaches besides Campbell and Fiske's traditional approach of inspecting correlation matrices (e.g., Eid, 2000 Eid, Lischetzke, Nussbeck, & Trierweiler, 2003 Kenny, 1976, 1979 Marsh, 1989 Marsh & Grayson, 1995 Saris & van Meurs, 1991 Widaman, 1985). This is mainly due to the fact that SEM...

Characteristics Of Sequential Observational Methods

As we define matters here, coding schemes are a central defining characteristic of sequential observational methods. Sometimes it useful to use the phrase systematic observation to distinguish the sorts of methods we are talking about from simply looking at behavior or producing narrative, journalistic reports. Then a brief definition of systematic observation might be the application of predefined coding schemes to sequences of live or recorded behavior (or transcripts of behavior) based on...

Examples of Multimethod Assessment Interpersonal Conflict and Aggression

William Graziano and his colleagues have conducted a program of research that also illustrates well the strengths and advantages of a multimethod approach in social psychology. We chose this particular study (Graziano, Jensen-Campbell, & Hair, 1996) to describe in some detail because it is a good example of all three forms of multimethod research multi-method assessment of the dependent variables, between-method replication of the independent variable, and...

Personality Is a Multilevel Phenomenon

A key component of our neosocioanalytic perspective on personality is that the domains of traits, motives, abilities, and narratives can be differentiated in hierarchical terms (see Hooker, 2002 Hooker & McAdams, 2003 Mayer, 1995 Roberts & Pomerantz, in press). For example, at the broadest level of the trait domain one finds the personality traits found in standard omnibus personality inventories. These are often the traits that make up the now ubiquitous measures of the Big Five. The...

Examining Stability Autoregressive Models

Autoregressive models are used to examine the stability of the relative standing of individuals over time. Figure 21.1 illustrates an autoregressive model for a three-wave data set. In this data set (Biesanz, West, & Millevoi, 2004), 188 college students were assessed at weekly intervals on a measure of the personality trait of conscientiousness (Saucier & Ostendorf, 1999). According to Saucier and Ostendorf, conscientiousness is comprised of four closely related facets orderliness,...

Multicohort Multioccasion Designs to Cross Validate Developmental Trends

Marsh (1998 also see Baltes & Nesselroade, 1979) argued that multicohort-multioccasion designs provide a stronger basis for assessing developmental self-concept differences than a typical cross-sectional methodology (comparisons of different age cohorts collected on a single occasion) or a true longitudinal methodology (comparisons of responses by a single age cohort collected on multiple occasions). In particular, the juxtaposition of the age effects based on the (cross-sectional) age...

MTMM Extensions The Multiple Indicator Approach

The Campbell-Fiske guidelines are frequently criticized for being based on correlations among observed variables rather than among latent constructs. Ironically, in the typical CFA MTMM approach, a single scale score often an average of multiple items is used to represent each trait-method combination. Marsh (1993b Marsh & Hocevar, 1988), however, argued that it is stronger to incorporate the multiple indicators explicitly into the MTMM design. When multiple indicators are used to represent...

Trait Source and Error Variance

Trait (construct) variance represents the systematic variance in a specific manifest variable associated with a particular latent trait, whereas source variance represents the systematic variance in a specific manifest variable associated with a particular latent source. The error variance in a specific manifest variable involves two different aspects residual systematic variance (i.e., reliable variance not associated with trait and source factors) and nonsystematic effects (i.e., measurement...

Thematic Content Analysis

Thematic content analysis is used here as a summary label for a number of approaches that have been developed in the context of motivational psychology (Smith, 1992). Generally, these approaches have human judges identify critical thematic references in a text. Ratings are made either each time a theme occurs or as global ratings reflecting the prevalence of a theme across an entire text. In either case, the analyses are based on standardized coding systems that define a psychological construct...

Other Informants Child and Infant Temperament

Probably one of the most common forms of assessment in developmental research is the use of questionnaires or interviews that assess some aspect of a participant's functioning from someone else's perspective, such as that of a parent, teacher, or peer (Neyer, chap. 4, this volume). Paper-and-pencil measures are often used because of their efficiency. Many participants can be questioned at one time, so obtaining these reports is much less time consuming and less expensive than observational...

Nonreactive Measures as Distinct Techniques

In the literature, the term nonreactive measurement is used in at least two senses, in a dichotomous as well as a continuous sense. Textbooks often think of nonreactive measures as representing a distinct set of procedures sharply different from reactive methods. Actually, there is a comparatively stable core of measures that are commonly subsumed under this heading. In social psychology, for instance, one of the most cited and consequently most prototypical nonreactive method is the so- called...

Multiple Methods and Personality Traits

As we noted, one of the persistent disputes in personality psychology is between those who believe that self-reports or observer methods should hold priority in the field. The programmatic efforts of David Funder and his colleagues demonstrate that multiple methods bring multiple perspectives to our efforts to understand the behavioral manifestation of personality traits. For example, people judging the behaviors of others perceive different cues as more relevant to personality than the...

Evaluating the Constructs Underlying Ability Achievement and Other Tests of Cognitive Competencies

It is important to appreciate that the vertex of the hierarchical model (derived factor analytically) and 2To be sure, other frameworks have been proposed, involving emotional, moral, multiple, and practical intelligence but these have yet to generate meaningful empirical advances beyond what conventional cognitive ability assessments afford (cf. Brody, 2003 Gottfredson, 2003b Hunt, 1999 Lubinski & Benbow, 1995). Messick (1992) in particular has skillfully demonstrated that many of these...

Generalizability Theory GT

Generalizability Theory (Cronbach, Rajaratnam, & Gleser, 1963 Cronbach et al., 1972 Gleser, Cronbach, & Rajaratnam, 1965 Shavelson & Webb, 1991) combines Brunswik's request for representative multifacet designs with the true-score model of Classical Test Theory (CTT Lord & Novick, 1968 Spearman, 1910). Like CTT, GT assumes that each person (or other object of-measurement) has a true score on the measured attribute. In GT, this score is called the universe score. Whereas CTT treats...

The Criterion Problem Of Accurate Informant Assessment

In terms of Brunswik's lens model, accurate perception is characterized by the convergence of cue validity and cue utilization. When researchers examine the accuracy of informant assessment, they inevitably face the criterion problem. Kruglanski (1989b) discussed three distinct notions of accuracy criteria used throughout the literature (i.e., the correspondence between a judgment and one or more independent indicators of the psychological construct, interpersonal consensus, and pragmatic...

How Are the SRM the WAM and the RAM Different

The described models, the SRM, the WAM, and the RAM, sometimes complement and sometimes compete with each other. The models are more or less based on Brunswik's approach and share the assumption that interpersonal perception should be observed in real settings. Each of them has contributed to the revival of interpersonal perception research and has yielded important, potentially useful insights when using informant assessment in research. The SRM is a statistical model that enables data to be...

Nonreactive Methods In Psychological Research

Participants in social science research usually think. That is, they interpret the actions of a researcher and relate it to their own beliefs, emotions, and intentions. Accordingly, the behavior they exhibit during investigations is controlled by what they think is appropriate, depending on their interpretation of the study situation and their motivation to comply with these assumed requirements. Hence, what is often measured is not a natural state but the participant's intentional...

Latent Trait Effects Outcomes Required for Construct Validity

In the first example Figure 27.1 , the manifest variables were the individual symptom ratings. More typically, however, these procedures use summary scores as manifest variables e.g., the summary score for the nine ADHD-IN, nine ADHD-HI, and eight ODD symptoms for the four sources , the focus of our second example. Figure 27.2 shows the model for the summary scores. This model involves three latent trait factors ADHD-IN, ADHD-HI, and ODD and four latent FIGURE 27.2. Heuristic representation of...

Advantages Of Webbased Methods

One of the principal reasons why Web-based methods are so popular is the fundamental asymmetry of accessibility What is programmed to be accessible from any Internet-connected place in the world will surely also be accessible in a university laboratory but what is programmed to work locally may most likely not be accessible anywhere else. A laboratory experiment, for instance, cannot simply be turned into a Web experiment by connecting the host computer to the Internet. But any Web experiment...

The Realistic Accuracy Model RAM

Realistic Accuracy Model

The realistic accuracy model RAM by Funder 1995 begins with the premise that personality traits are real and observable. As a consequence, the RAM assumes that informants reach consensus not because they share similar meaning systems or because of overlap, but rather because their judgments about a target's personality are at least partly accurate. According to the RAM, the path between a target's personality and the accurate informant judgment can be described in four steps, each associated...

P

Indicates how kappa is computed .84 in this case . Agreement expected due just to chance is subtracted from both numerator and denominator, thus kappa gives the proportion of agreement corrected for chance. Exact second-by-second agreement may be too stringent. Given human reaction time and other considerations, investigators may be willing to permit their coders some tolerance, which the GSEQ program allows. For the tallies in Figure 10.3, a 2-second tolerance was specified, thus agreements...

Convergent and Discriminant Validity

Given the lack of knowledge about the true attributes of objects, convergence across different methods for the same attribute is often the best alternative. This type of validity has been called convergent validity Campbell amp Fiske, 1959 . Demonstrating convergent validity is not sufficient, however, because convergence alone does not yet guarantee that the methods measure what they should measure. It only shows that the methods measure the same factors. As previously outlined, some or even...

Multipleindicator Models

In contrast to single-indicator models, multiple-indicator models are able to separate measurement error from trait-specific method influences. Moreover, the hypothesis that method effects are trait specific and do not perfectly generalize across traits can be statistically tested. We will only describe three multiple-indicator extensions a general model that is able to estimate the latent correlations between different trait-method units, a model that is related to the CTCU model, and a model...

Analysis of Variance Generalizability Theory and Multilevel Modeling

The application of analysis of variance ANOVA models has a long tradition in multimethod research. To analyze the conver gence of several methods measuring the same trait, ANOVA models are routinely applied Mill-sap, 1995b Tinsley amp Weiss, 2000 . In general, two types of factors can be considered in ANOVA models random and fixed factors. Random factors are considered when the levels of a factor are a random sample from a population and the research goal is to generalize...

Linguistic Inquiry and Word Count

Linguistic Inquiry and Word Count LIWC Pennebaker et al., 2001 was originally developed in the context of Pennebaker's work on emotional writing. It was designed to reveal aspects of writing about negative life experiences that predict subsequent health improvements Pennebaker amp Francis, 1996 Pennebaker, Mayne, amp Francis, 1997 . More recently LIWC has been used to analyze language use in a wide variety of text sources including literature, personal narratives, press conferences, and...

Section Ii Assessment Methods

Baird, Brendan M. CHAPTER 5 Stone, Arthur A. Litcher-Kelly, Leighann CHAPTER 6 Reips, Ulf-Dietrich Web-Based Methods pp 73-85 CHAPTER 7 Drasgow, Fritz Chuah, Siang Chee Computer-Based Testing pp 87-100 CHAPTER 8 Lubinski, David Ability Tests pp 101-114 CHAPTER 9 Robinson, Michael D. Neighbors, Catching the Mind in Action Implicit Methods in Personality Research and CHAPTER 10 Bakeman, Roger Gnisci, Augusto Sequential Observational Methods Quantitative Text Analysis...

Betweenmethod Replication

When social psychologists think of multiple methods, we are perhaps just as likely to focus on the independent variable as the dependent variable. There is clear virtue even in an exact or strict replication of a finding e.g., Hendrick, 1990 , but social psychologists prefer a conceptual replication, using a different operationalization of the independent variable e.g., Gerard amp Mathewson, 1966 Pratka-nis, Greenwald, Leippe, amp Baumgardner, 1988 . If alternative manipulations replicate the...

Emotion Assessed Through Language

Self-report measures, where participants provide an evaluation of their emotional experience, form the most diverse yet most widely used set of assessment tools for measuring emotion Larsen amp Fredrickson, 1999 . Measures range from rating scales and adjective checklists, to analog scales and real-time rating dials. Proponents of self-report measures e.g., Baldwin, 2000 assume that participants are in a privileged position to monitor, assess, and integrate...

Emotion Assessed Through Physiology

Emotion output that can be assessed with physiological methods can be divided into two categories the somatic changes and changes reflecting autonomic or central nervous system activity. The somatic changes most useful to emotion researchers concern muscle movements associated with emotional expression, particularly those somatic changes on the face. Measures of somatic change. One useful measurement strategy is to have an observer rate how much emotion a target participant appears to be...

Emotion Assessed Through Behavior

Behaviors that are linked to emotions range from the very simple, such as defensive reflex actions, to the complex, such as sequences of action tendencies. Emotions likely evolved to produce adaptive actions, such as to approach desired objects or to withdraw from dangerous objects, as well as to support more flexible action tendencies associated with survival. Researchers may take advantage of these behavioral outputs to estimate emotions. Behavior action tendencies. One behavioral...

The Organization Of Cognitive Abilities

There are literally hundreds of ability tests Carroll, 1993 Cattell, 1971 Jensen, 1980, 1998 Sternberg, 1994 ,1 and a framework is needed to organize them. Over the years, proposals have been made to Support for this article was provided by a Templeton Award for Positive Psychology, a NICHD Grant P30HD15052 to the John E Kennedy Center at Vanderbilt University, and a Cattell Sabbatical Award. Earlier versions of this manuscript profited from many excellent suggestions by Camilla P Ben-bow....

Introduction To The Basic Item Response Models

Item response theory IRT see Baker, 1992 Hamble-ton amp Swaminathan, 1989 usually is considered a class of statistical models for categorical data to which the Rasch model RM and the two-parameter logistic model belong, but not latent class analysis LCA . However, there are at least two reasons why LCA can and should be seen as an IRT model. Whereas the Rasch model tries to measure a latent trait of the persons, LCA tries to identify latent classes of persons. Both measure a latent variable,...

Some Alternative Metrics

In an ideal world of construct validity, we would collect data whose assessment methods share a minimum of facets e.g., structured vs. semistruc- tured vs. unstructured, paper-and-pencil vs. observations by others, verbal vs. behavioral, accessible vs. inferred by observers, transparent vs. masked with other assessments of these constructs. As this book details, other areas of psychology have developed metrics that could and should be applied to I O research. These other assessments could...

Temporal Duration Emotions as States Emotions as Traits

Emotions are dynamic processes that take place over time often rapidly, but sometimes gradually involving a cascade of different response systems, such as those mentioned earlier. Each response system has its own dynamic and duration. For example, the central nervous system is very fast with negative stimuli producing changes at around milliseconds Smith, Cacioppo, Larsen, amp Chartrand, 2003 , the cardiac system somewhat slower with changes taking place on a beat-by-beat basis , the skin...

Weintraubs Analysis of Verbal Behavior

Weintraub's 1981, 1989 work on verbal mannerisms was inspired by the clinical observation that individuals speaking under stress often reveal important information about their psychological adjustment. Drawing on his medical training and practice, Weintraub argued that psychological defense mechanisms manifest themselves in speech patterns obtained under mildly stressful conditions. He assessed these defense mechanisms from the language that participants spontaneously use when they talk for 10...

Wextor

First, we use WEXTOR Reips amp Neuhaus, 2002 , a Web service, to create, store, and visualize experimental designs and procedures for experiments on the Web and in the laboratory. WEXTOR dynamically creates the customized Web pages needed for the experimental procedure. It supports complete and incomplete factorial designs with between-subjects, within-subjects, and quasi-experimental natural factors, as well as mixed designs. It implements client-side, response time measurement and contains a...

Text Analysis Example From Social Psychology

How do people respond to physical symptoms, and what makes them decide whether or not to seek treatment The methodological toolbox in psychology is large, and there are a number of potential ways to address this question. Yet the default strategy has been to rely on just one tool the questionnaire. In this case, for example, a researcher might create a health-decision questionnaire consisting of a number of Likert-scaled items, such as How serious do your symptoms have to be before you see a...

How Far Can We Go With Counting Words

Given that word count-based measures possess rather good psychometric properties, how far can we go with counting words Frequently researchers voice their scientific disdain for text analysis programs that are unable to distinguish between sentences as simple as the dog bit the man and the man bit the dog Hart, 2001, p. 53 . Its blindness to context makes word-count approaches sometimes appear painfully dumb. Not only are they unable to pick up irony or sarcasm e.g., Thanks a lot, accompanied...

Understanding the Question

When an individual responds to a self-report measure, he or she must first make sense of the question being asked Schwarz, 1999 Tourangeau et al., 2000 . To do this, the respondent must understand the literal meaning of the question, and anything that impedes this understanding e.g., vague or unfamiliar words, complicated sentence structure will undermine the quality of the self-report measure. Psychological assessment and survey methodology textbooks suggest that to avoid misunderstandings,...

Multilevel Models for Generalizability Analysis

Generalizability theory can be viewed as a special case of multilevel analysis. In the one-facet nested design, the nesting structure is clear The items are nested in the persons. In the one-facet crossed design, the nesting structure is arbitrary Items can be seen as nested in persons or persons nested in items. Both specifications lead to the same results. Because of the analogy with the nested design, we will use the specification structure of items in persons. In a two-facet design, the...

In Search Of Depressogenic Thought Processes

A variety of studies using a variety of cognitive methods have sought to discover the depressogenic thought processes responsible for depression for reviews, see MacLeod amp Mathews, 1994 Segal, 1988 Segal amp Ingram, 1994 . However, one problematic result emerges from this research. Specifically, it is extremely difficult to distinguish formerly depressed participants from never depressed participants on measures of cognitive bias e.g., Segal amp Ingram, 1994 . That is, depressive biases in...

Analysis of Artistic Change Martindales Regressive Imagery Dictionary

To identify regularities underlying changes in artistic work over time, Martindale 1990 developed a word count program that is based on the Regressive Imagery Dictionary. Martindale's 1990 theorizing starts from the observation that artistic work shows a steady increase in complexity over time. He explains this increase by drawing on two funda mental psychological processes humans' preference for medium levels of arousal and hence moderately complex sensory input and the physiological mechanism...

Campbell and Fiske

Brunswik Lens Model

Multimethod thinking in psychological assessment was influenced most strongly by the seminal paper of Campbell and Fiske 1959 . No other publication so importantly shaped researchers' awareness of the crucial role multimethod designs play in the construction and validation of measurement instruments Shrout amp Fiske, 1995 . Although Campbell and Fiske 1959 did not make reference to Brunswik's work, their proposals were guided by similar insights and ideas. Campbell and Fiske 1959 introduced the...