Computerbased Testing

Fritz Drasgow and Siang Chee Chuah

The history of human advancement has shown few innovations with the wide-ranging impact of the computer. The ENIAC, or Electronic Numerical Integrator and Computer, was the first general-purpose electronic computer (Rojas, 2001). Perhaps the greatest asset of the ENIAC and successive generations of multifunction computers is their adaptability to various tasks. As described in this chapter, the computer's flexibility has provided a great advantage for psychological measurement.

Significant improvements have been made to the computer since the days of the ENIAC. In 1965, Gordon Moore predicted that computer processor speed would double every 18 months. This prediction has proved remarkably accurate: Processor speed has indeed doubled at this rate and shows no signs of slowing down. Computers have advanced from paper tapes and punch cards to sophisticated graphic user interfaces (GUIs) and from teletypes and line printers to virtual environments. Computers no longer cost millions of dollars and fill large rooms. Testing programs can now afford computers that fit conveniently on desktops and easily transported laptops that allow in situ assessment. It is no surprise, therefore, that computer usage has grown exponentially.

Computers have been used in many ways for psychological measurement. Perhaps most straightforward is the computerization of tests previously developed for paper-and-pencil administration. Such tests offer little in terms of improved psychometric properties but can have important administrative advantages.

More sophisticated are tests that adapt their difficulty level to match the ability of the examinee. Computer adaptive sequential tests (CASTs; Luecht & Nungester, 1998), computerized adaptive tests (CATs; Sands, Waters, & McBride, 1997), and shadow tests (van der Linden, 2000) provide examples. These tests generally use traditional multiple-choice questions, but use the computer to strategically select items of appropriate difficulty for each individual examinee. By omitting items that are too easy or too hard for a particular individual, test length can be substantially reduced with no loss of measurement accuracy.

Computerization also allows more radical innovation in assessment. Consider that in many situations, tests are designed to predict future behavior. For example, academic admissions tests are intended to predict performance in college, employment tests are used to predict job performance, and licensing and credentialing tests seek to assess whether candidates have the requisite skills to practice their profession without harming the public. Campbell's (1990) model suggests that performance is a function of declarative knowledge, procedural knowledge, and motivation. Declarative knowledge consists of an individual's repertoire of facts, rules, and principles; procedural knowledge involves knowing how to perform a task; and motivation concerns the amplitude, direction, and persistence of one's efforts. Based on their success in predicting performance (Kuncel, Hezlett, & Ones, 2001; Schmidt & Hunter, 1998), it appears that traditional multiple-choice items do a good job of assessing declarative knowledge. However, there is a growing consensus that multiple-choice items are inadequate for assessing procedural knowledge.

Computer-based testing provides an opportunity to improve the assessment of an individual's ability to perform by simulating tasks that are important in the workplace or classroom (Computerization Implementation Committee, 2001). For example, consider the job of a certified public accountant (CPA). Multiple-choice items can accurately assess an individual's repertoire of accounting facts, rules, and principles. However, the American Institute of Certified Public Accountants added simulations of client encounters to their licensing exam in 2004 because declarative knowledge appears necessary, but not sufficient, to ensure competence as a practicing accountant. More specifically, the exam now attempts to assess whether a CPA candidate can successfully solve the vague and ill-defined problems posed by clients. A computer simulation of a client encounter can simultaneously test knowledge of accounting facts, rules, and principles as well as the application of that knowledge to the problems presented by clients (Computerization Implementation Committee, 2001). In this way, computer-based testing provides an opportunity for more comprehensive assessment.

Computerized assessment can also broaden the domain of assessment by incorporating multimedia stimuli. Vispoel (1999), for example, used the computer's ability to present sound clips to assess musical aptitude, Olson-Buchanan et al. (1998) incorporated video clips in their computerized assessment of conflict resolution skills, and Acker-man, Evans, Park, Tamassia, and Turner (1999) displayed images on a high-resolution color monitor to assess medical students' ability to diagnose der-matological skin disorders.

In sum, there has been a proliferation of research designed to explore and exploit opportunities provided by computer-based assessment. This chapter provides an overview of the diverse efforts by researchers in this area. It begins by describing how paper-and-pencil tests can be adapted for administration by computers. Computerization provides the important advantage that items can be selected so they are of appropriate difficulty for each examinee. Some of the psychometric theory needed for computerized adaptive testing is reviewed. Then research on innovative computerized assessments is summarized. These assessments go beyond multiple-choice items by using formats made possible by computerization. Then some hardware and software issues are described, and finally, directions for future work are outlined.

Was this article helpful?

0 0

Post a comment