response or the (high) difficulty of giving a free response. On the other side, the third method in this hypothetical example, the numerical response format, interacts with the content area depending on which formula has to be applied and on the choice of the numbers. An easy numerical calculation makes the items easier, whereas the application of a complex formula and/or fractions can make the items rather difficult.

From this perspective, the main effects and the interaction model are not distinct types of models, but merely the extreme variants of a whole spectrum of models allowing for more or less interactions between the items and the methods. But even with the highest degree of interactions between items and methods, the interaction model is latent additive with respect to the person parameter 0v; in other words, these models assume only one trait parameter for all items and methods, and hence, they remain unidimensional.

Interaction between persons and methods.

When a test instrument has been administered by applying different methods, this may not only have implications for the item difficulties, but also for the latent traits used by the persons. Method A, for example, items with an open response format that require the respondents to verbalize the solution of a task, may address the respondents' verbal and creative abilities. The "same" items with a multiple-choice response format, in contrast, address the person's ability to make "good guesses" by some distracter elimination technique. Hence, methods may be associated to different traits or latent dimensions.

This is also a kind of interaction but an interaction between methods and persons. These models may be called ability models as compared to difficulty models. They can be formalized in the very same way as difficulty models, that is, by introducing double-indexed parameters 6v. instead of latent additive (single-indexed) person and method parameters. The multidimensional multimethod model can be formalized as exp(0. -<7.)

where öyj is the trait parameter of person v if method j is used. The test items have the same difficulty irrespective of the method applied, but the persons respond by means of a different trait depending on which method has been used. This model belongs to the family of multidimensional Rasch models that has been described by Adams, Wilson, and Wang (1997; see also the computer program ConQuest; Wu, Adams, & Wilson, 1997) and Rost and Carstensen (2002; see also the computer program MULTIRA; Carstensen & Rost, 1998). Model 8 is a submodel of these generalized Rasch models, which is defined by the following properties.

■ It is a between-item multidimensionality model, which means that the latent dimensions are specific for nonoverlapping subgroups of items, so that ever)7 item belongs to one and only one dimension.

■ The item parameters a remains the same for all methods.

■ It is a kind of facets model that refers to a test design where each item is a specific combination of two facets, for example, a content facet and a process facet or, as in our case, the item content and the administration method. In contrast to the general case of facets models (Rost & Carstensen, 2002), Model 8 is not symmetric with respect to the two facets: The item content facet is assumed to have a main effect only on the item difficulty, whereas the method facet is assumed to interact with the trait.

As in the case of the LLTM and the simple main effects model, the model can be specified by means of design matrices and, hence, is embedded in a more general model structure. This general multidimensional model is the multidimensional random coefficients multinomial logit model (MRCMLM; Adams, Wilson, & Wang, 1997; see also the computer program ConQuest; Wu, Adams, & Wilson, 1997) in which two design matrices, A and B, are used for separately specifying the component structure of the item difficulties (as in the LLTM) and a (different) component structure defining the latent traits.

y W} d vij where m is the index across all tasks (i.e., items X methods), h is the index of the components defining the traits, and g is the index of the components defining the difficulties. Both matrices, A and B, can be different and are different in our case. In fact they must be different to avoid identification problems.

This is illustrated for the example of four items and three methods, that is, 12 physical items, each four from three different methods in Table 18.4. The number of rows in both matrices must be identical because they refer to the same data, that is, to the same physical items. The columns, however, are different. There are three for the method-specific traits (Matrix A) and four for the method-free item contents.

One benefit of defining a multimethod model by means of design matrices is the flexibility of model specification. Again, incomplete test designs can be handled by the design matrices, and models can be specified that mix assumptions from different models.

As a multimethod model, Model 8 can be considered a model with weak assumptions for the traits (each method defining its own trait), but with strong assumptions regarding the items because they have the same difficulties under all methods.

Two Different Design Matrices for the Ability and the Difficulty Structure

Was this article helpful?

0 0

Post a comment