Alternative Approaches to Adaptive Testing

Several alternative approaches to CATs have been suggested. Two are briefly described next.

Testlets. Wainer and Kiely (1987) suggested the use of what they termed "testlets" as an improvement on adaptive testing as described earlier. Testlets are "the coagulation of items into coherent groups that can be scored as a whole" (Wainer, Bradlow, & Du, 2000, p. 246). Perhaps the most common version of a testlet consists of a reading passage followed by four to six questions. Wainer, Bradlow, and Du noted several reasons for using testlets. First, and perhaps most important, is that the traditional multiple-choice test composed of many short questions has been criticized as producing a form of assessment constituted by "decontex-tualized items" that are "abstracted too far from the domain of inference" (Wainer et al., 2000, p. 245). Because a testlet consists of a stem and several interrelated questions, a meaningful context is created. A more pragmatic advantage of testlets is that reading a passage or studying a spreadsheet takes time; asking several questions improves the information yield per unit of time.

Testlets (or avoiding Wainer & Kiely's [1987] jargon, sets of items that refer to the same stem) violate a central assumption of IRT in that items within a testlet are ordinarily more highly corre lated than expected based on the assumption that items measure a single latent trait d. That is, from the perspective of psychological measurement, five independent items provide more information about an individual's 0 than five dependent items referring to a common stem. Thus, analyzing tests consisting of testlets via a standard IRT model such as the three-parameter logistic may produce misleadingly optimistic results about the accuracy of measurement. Wainer et al. (2000) provided an appropriate psychometric model for tests composed of testlets and conducted a simulation study that demonstrated its effectiveness.

CAST. Computer-adaptive sequential testing (CAST; Luecht & Nungester, 1998) also uses sets of items as the fundamental building block for a test. These sets of items are called modules, and the collection of modules constituting a test is called a panel. Figure 7.3 illustrates a 1-3-3 panel. All examinees would be administered Module 1M, which might contain 15 to 20 items and be moderately difficult. Examinees with high, moderate, and low scores would be branched to Modules 2H, 2M, and 2E, respectively; ordinarily about one third of the examinees would be branched to each stage 2 module. In stage 3, examinees who completed Module 2M would be routed to Module 3E, 3M, or 3H, depending on whether their scores were low, moderate, or high. Cut scores for routing would be set


Was this article helpful?

0 0

Post a comment