## Multivariate Multilevel Measurement

Raudenbush et al. (1991) extended the measurement model by combining items from several different scales in one analysis. The constant in the multilevel model is then replaced by a set of dummy variables that indicate to which scale each item belongs. This is similar to a confirmative factor analysis, but with the restriction that the loadings of all items that belong to the same scale are equal and that there is one common error variance. These are strong restrictions, and multilevel structural equation modeling (Hox, 2002) is both more flexible and less restrictive. However, multilevel structural equation modeling does not model raw scores; it is based on simultaneous analysis of a person-level and a group-level covariance matrix. Therefore, it does not produce estimated scores on the latent variables. Consider the following example of combining individual-level and group-level information. Assume we ask pupils in 100 classes to rate their teacher using the semantic differential method (Hoyle et al., 2002). In the semantic differential method, three factors—"evaluation," "activity", and "potency"—are assumed to underlie a set of bipolar rating scales. In our example, each teacher is rated by the students on a set of three items each for evaluation, activity, and potency. In addition, the teachers rate themselves on the same set of nine items, using the same bipolar rating scale that runs from -4 to +4. This creates a multitrait-multimethod structure where the three semantic differential factors are the traits, and the teacher and students are the measurement methods. In addition, we have multiple raters for the student ratings with generally a different number of student raters for each teacher.

The resulting data can be viewed as a multilevel structure, with nine items varying on both the pupil level and the teacher level, and nine items varying only on the teacher level. One convenient way to model data such as these is to use a multivariate multilevel model with separate levels for the items, the pupils, and the schools. At the lowest level we have 18 items, which refer to three semantic differential scales for the pupils and three for the teachers. Thus, we create 6 dummy variables, dpjJ (dUj to d6jj) to indicate the 3 scales x 2 types of raters, exclude the regression coefficient for the intercept from the model, but keep the lowest level variance term to estimate the residual variance among the 18 items. Hence, at the lowest level we have

Y... = n,..d... + n7..d... + ... + jt,..d,.. +e.„. (10)

and at the class/teacher level (the third level in the multivariate model), we have

By substitution, we obtain the single equation version

+u, d, + U- d^.. + ... + m d, + e 1 j 1 tj 2 j 2lj 6 j 61] hi]

The model described by Equation (13) provides us with estimates of the six scale means and of their variances and covariances at the pupil and class level. Because we are mostly interested in the variances and covariances in this application, RML estimation is preferred to FML estimation. Table 19.2 presents the RML estimates of the covariances and the corresponding correlations at the pupil level and at the school level.

Table 19.2 shows that most of the variance is between classes. The variances and covariances at the class level are important for inspecting the convergent and discriminant validity of the measures. In fact, at the class level we have a multitrait-multi-method matrix that consists of three traits and two methods. The pairwise correlations between the three methods measured both through pupils and teachers is the validity diagonal. The correlations of