FIGURE 18.6. Loading plot of the first two principal components based on the latent correlations.

nearly the same loadings, that is, they form a kind of cluster in this two-dimensional space. The fact that 94% of the variance is explained by the first two components and the evident interpretation of the results provides support for the assumption that two latent dimensions would suffice to explain the data. With regard to the task difficulties, it makes no difference whether the unidimensional or multidimensional Rasch model is applied because the item difficulties of the unidimensional and the seven-dimensional model are approximately equal.

Finally, the results concerning the mixed Rasch model are to be discussed. Two different solutions were calculated: a two-class and a three-class model. The two-class model with 146 parameters has a log likelihood value of-12,169, whereas the three-class model with 219 parameters has a log likelihood value of-11,976. Although both values are smaller than the seven-dimensional Rasch model's log likelihood value, because of the large number of parameters of the mixed Rasch model, their B1C values (25,091 and 25,383 for the two-class and three-class models, respectively) are larger than that of the seven-dimensional Rasch model (24,910, see Table 18.5).

To illustrate the results of the two-class model, Figure 18.7 shows 10 diagrams—one diagram for each content area—that present the item parameters for each class of the two-class model. It turns out that the profiles of item parameters in both classes are, despite some deviations, more or less parallel. The irregularities, however, do not relate to one and only one of the cognitive components and, hence, are not specific to one method. The component "dealing with numbers," for example, seems to be the only item that makes a difference between the two classes in the content areas "respiration and photosynthesis" and "predator and prey." The use of mental models when solving a task is responsible for the existence of two classes (instead of one) in the areas "energy" and "waste."

An inspection of all 10 diagrams in Figure 18.7 provides no evidence that there is a systematic interaction among items (contents), methods (cognitive components), and persons. There is, however, a first-order interaction between methods (components) and contents (items) because the profiles shown by the 10 diagrams are rather different. The distinction of two classes of persons is "only" needed for taking account of some content-specific method effects that seem to indicate some deficiencies in item construction rather than a systematic persons-methods interaction.

Summarizing the findings of all multimethod models considered so far, it can be concluded for the PISA 2003 field trial data that

■ There is a strong interaction between the content and the method, which is indicated by the superiority of all models assuming such an interaction (Models 7, 10, and 15) as compared to those that do not (Models 5 and 8).

■ There is an interaction between methods and persons (7D LLTM, Model 8, and seven-dimensional Rasch model, Model 10) that can be represented by a multidimensional methods model, whereas creating an own method factor for each of the seven cognitive components would obviously be an overparameterization (two factors might be sufficient).

■ There is no systematic interaction between contents (items) and persons, which is indicated by the more or less parallel item profiles of the two latent classes in the mixed Rasch model (Model 15).

Was this article helpful?

0 0

Post a comment