Although adapting conventional tests for computerized administration has many procedural advantages, the psychometric properties of the tests are not improved. Specifically, highly capable examinees answer easy items correctly with high probability, and weak examinees answer difficult items at near chance levels. As described in this section, the computer's dynamic capabilities can be used to selectively administer items to examinees so that the items are of appropriate difficulty for each individual and thereby provide useful information about the respondent's ability level.
In its simplest form, a computerized adaptive test (CAT) begins by administering an item of moderate difficulty. If the examinee answers correctly, the computer branches to an item of greater difficulty; if the examinee answers incorrectly, the computer branches to an easier item. After the second item is answered, the computer again branches to a more difficult or easier item, depending on whether the answer was correct or incorrect. This process continues, and ordinarily the computer rapidly homes in on the examinee's ability level.
By targeting items to examinees' ability levels, it is possible for a test to provide more precise assessment with fewer items: A reduction in test length of approximately 50% might be expected. However, testing time is not usually reduced by this amount because examinees tend to take longer answering items of appropriate difficulty than items that are too easy or difficult. Nonetheless, substantial reductions in test length and moderate reductions in testing time can be achieved with no loss of measurement precision.
Adapting item difficulty has a derivative benefit for testing programs with important consequences for examinees (i.e., "high-stakes" tests). Ordinarily, high-stakes tests are administered only a few times per year. For example, the conventional CPA licensing exam was administered only twice per year (in May and November). Such tests are offered infrequently because a new form must be created for each administration to eliminate any opportunity for cheating. Obviously, developing a new form for every administration requires a great deal of time, effort, and expense. By using the computer to adapt item difficulty, each examinee receives a unique test form. Consequently, with item exposure controls in place (see following), test security is maintained, and continuous test administration is possible.
Thus, examinees can go to a testing program's Web site and schedule their exam at a time convenient for them. For example, the CPA licensing exam allows candidates to schedule their test at any time during a 2-month window within every quarter of the year. Obviously, this convenience is greatly appreciated by examinees. In sum, using the computer to adapt test difficulty allows tests to be shorter in length, take less time, and can be scheduled at times that are convenient to examinees, yet maintain test security
Was this article helpful?