Joop Hox and Cora Maas
Theoretical constructs used in social and behavioral science are often complex, and they have an indirect relationship to the corresponding empirical observations. The distance between a theoretical construct and its observable phenomena can create problems for researchers, causing them to explicitly state how they plan to measure what they are theorizing about. To ensure construct validity, the methodological advice often given is to measure each construct in more than one way (e.g., Hoyle, Harris, &Judd, 2002; Kerlinger, 1973). Fiske (1971) advocated not only using multiple opera-tionalizations of each construct, but also purposefully manipulating operationalizations to span different theoretical perspectives and modes of assessment. This raises questions about the convergence and discriminability of different constructs and measures, which underlies the development of the multitrait-multimethod method (Campbell & Fiske, 1959).
This chapter focuses on using multilevel modeling to combine information from different sources and on assessing the reliability and validity of the resulting estimates. It starts with a brief introduction to multilevel analysis. Following this introduction, three measurement approaches are discussed where multilevel modeling is a valuable and effective analysis tool: facet design, assessing contextual characteristics, and generalizability theory. These approaches were chosen because they all, each in their own fashion, aim to incorporate information from several dis tinct sources in one measurement instrument. Each approach is explained using an example including an analysis of a small data set. The role of multilevel analysis in all three approaches is to assess the contribution of different sources of variance (due both to different traits and to the specific measurement modes used) in designs where standard analysis methods encounter difficulties.
Was this article helpful?