The purpose of innovation is to improve a test and not innovate for the sake of innovation. Jodoin (2003) compared innovative item types and traditional multiple-choice items and found that although innovative item types returned more information per item than multiple-choice items, the amount of time spent by respondents was disproportionately higher than the return. Thus, innovative item types were less efficient in terms of information yield per unit of time than multiple-choice items. So, what does innovation add to a test? Research by Bennett, Morley, and Quardt (2000) suggests that innovative item types broaden the construct measured by the test. They compared a test form consisting of their innovative item type with a test consisting of traditional items; they found that the disattenuated correlation between them was r = .70. This suggests some, but not complete, overlap between the latent traits assessed by the item types.
A further consideration is the extent to which a test authentically represents real-world skills required of the respondent. With the use of graphics, sound, and full-motion video, we can emulate many important real-world situations. Candidate performance on such realistic simulations is assumed to tap the same skills required by real-world situations and, therefore, constitutes a valid measure of an individual's performance in real-world situations. Therefore, innovative item types using new formats and video, graphics, or sound clearly broaden the skills assessed and improve the substantive richness of tests.
These innovative items can have very high face validity/authenticity. For example, some situational judgment tests used in preemployment testing use written passages that describe social situations. However, in the real world we are dependent on interpersonal cues such as verbal tone and facial expression for information regarding the situation. Full-motion video simulations are able to capture such subtle nuances and convey a more natural experience of the situation. On the other hand, written passages must rely on explicit description of otherwise subtle nuances to convey the situation. This may compromise validity because directing the attention of the respondent to particular cues makes it impossible to assess the candidate's ability to notice those nuances.
Was this article helpful?