The variance components are <5^ = 0.009, a] = 0.018, and G2residual - 0.232, which account for 3%, 7%, and 90% of the variance, respectively.

The variance components themselves are unstandardized; therefore, the interpretation uses the percentages. Three percent of the variance is associated with persons, 7% with items, and the remainder with the interaction and error.

The generalizability coefficient (G coefficient) for the preceding example depends on the decisions one wants to make (Shavelson & Webb, 1991). For relative decisions, only the variance component of the interaction between persons and items contributes to the measurement error. When the variance component is large, this means that the relative position of persons is different for the different items. Because all persons answer the same items, the item variance doesn't influence the relative position. In contrast, for absolute decisions, both the item variance and the variance of the interaction are important. In our example, the formulas for the estimated error variances are

where m is the number of items. Calculating the error variances yield 0.06 for both the relative vari ance and the absolute variance (the estimates are equal due to rounding). The formula for the G coefficient for relative decisions is a2

G - coefficient = —-———. (20) (< + </)

The calculated G coefficient is 0.134. The interpretation of this coefficient is analogous to the interpretation of the reliability coefficient in classical test theory. Because of the simple data set used, we refrain from further interpretation.

The reliability-like index of dependability for absolute decisions is formulated as

For this example the calculated index is 0.125. The interpretation of this coefficient is not exactly the same as the interpretation of the G coefficient, but broadly speaking it has the same function. In both cases, the generalizability coefficient indicates to what extent the measurements converge across specific method facets, including possible interaction effects. The decision across which method effects we need to generalize (which leads to different generalizability coefficients) remains, of course, with the researcher.

Was this article helpful?

0 0

Post a comment