John D Wasserman And Bruce A Bracken

PSYCHOMETRIC THEORIES 44 Classical Test Theory 44 Item Response Theory 44 SAMPLING AND NORMING 45

Appropriate Samples for Test Applications 45 Appropriate Sampling Methodology 46 Adequately Sized Normative Samples 46 Sampling Precision 47 Recency of Sampling 48 CALIBRATION AND DERIVATION OF REFERENCE NORMS 48 Calibration 48

Derivation of Norm-Referenced Scores 49 TEST SCORE VALIDITY 50

Internal Evidence of Validity 50

External Evidence of Validity 52 Validity Generalization 54

TEST SCORE RELIABILITY 54 Internal Consistency 55

Local Reliability and Conditional Standard Error 55 Temporal Stability 56 Interrater Consistency and Consensus 56 Congruence Between Alternative Forms 57 Reliability Generalization 57 TEST SCORE FAIRNESS 57 Terms and Definitions 58 Internal Evidence of Fairness 59 External Evidence of Fairness 61 THE LIMITS OF PSYCHOMETRICS 62 REFERENCES 62

"Whenever you can, count!" advised Sir Francis Galton (as cited in Newman, 1956, p. 1169), the father of contemporary psychometrics. Galton is credited with originating the concepts of regression, the regression line, and regression to the mean, as well as developing the mathematical formula (with Karl Pearson) for the correlation coefficient. He was a pioneer in efforts to measure physical, psychophysical, and mental traits, offering the first opportunity for the public to take tests of various sensory abilities and mental capacities in his London Anthropometric Laboratory. Galton quantified everything from fingerprint characteristics to variations in weather conditions to the number of brush strokes in two portraits for which he sat. At scientific meetings, he was known to count the number of times per minute members of the audience fidgeted, computing an average and deducing that the frequency of fidgeting was inversely associated with level of audience interest in the presentation.

Of course, the challenge in contemporary assessment is to know what to measure, how to measure it, and whether the measurements are meaningful. In a definition that still remains appropriate, Galton defined psychometry as the "art of imposing measurement and number upon operations of the mind" (Galton, 1879, p. 149). Derived from the Greek psyche

(^X^, meaning soul) and metro (^eTpt>>, meaning measure), psychometry may best be considered an evolving set of scientific rules for the development and application of psychological tests. Construction of psychological tests is guided by psychometric theories in the midst of a paradigm shift. Classical test theory, epitomized by Gulliksen's (1950) Theory of Mental Tests, has dominated psychological test development through the latter two thirds of the twentieth century. Item response theory, beginning with the work of Rasch (1960) and Lord and Novick's (1968) Statistical Theories of Mental Test Scores, is growing in influence and use, and it has recently culminated in the "new rules of measurement" (Embretson, 1995).

In this chapter, the most salient psychometric characteristics of psychological tests are described, incorporating elements from both classical test theory and item response theory. Guidelines are provided for the evaluation of test technical adequacy. The guidelines may be applied to a wide array of psychological tests, including those in the domains of academic achievement, adaptive behavior, cognitive-intellectual abilities, neuropsychological functions, personality and psy-chopathology, and personnel selection. The guidelines are based in part upon conceptual extensions of the Standards for

Educational and Psychological Testing (American Educational Research Association, 1999) and recommendations by such authorities as Anastasi and Urbina (1997), Bracken (1987), Cattell (1986), Nunnally and Bernstein (1994), and Salvia and Ysseldyke (2001).

0 0

Post a comment