The implementation of multimethod research strategies requires the knowledge of statistical approaches that consider the characteristics of the data inherent in multimethod strategies. These models must contend with the fact that an observed variable not only reflects the construct under consideration but also method-specific influences. Consequently, each measured value can be decomposed into a component that reflects the construct and is shared with other methods, as well as a component not shared with other methods. This method-specific component includes not only systematic method-specific influences, but also unsystematic measurement error. To separate true measurement error from systematic method-specific influences, appropriate methodological approaches are needed. Only by separating unsystematic measurement error from method-specific effects may one evaluate the degree to which the unique part of a measure reflects unsystematic measurement error versus systematic method-specific influences. Hence, data analytic procedures can be classified into methods that allow a separation of method-specific and error-specific influences and those that do not. Moreover, some data analytic approaches focus on the multimethod analysis of one construct, whereas other, more elaborated approaches, consider several methods measuring several constructs. Only the latter approach allows a systematic analysis of the generalizability of method effects across constructs (e.g., whether the bias of a rater is the same for all constructs being considered or whether a rater-bias is construct-specific). Hence, data-analytic procedures can be classified into approaches that allow analyzing the generalizability of method effects across traits and those that do not. Finally, data-analytic approaches can be classified according to the nature of the data being analyzed. Methodological approaches developed for metrical variables are usually not appropriate for categorical variables and vice versa.
This handbook gives an overview of advanced statistical approaches for analyzing multimethod data. Models for categorical data include classical approaches like Cohen's (Bakeman & Gnisci, this volume, chap. 10; Nussbeck, this volume, chap. 17) as well as more advanced methods like log-linear models (Nussbeck, this volume, chap. 17) and models of item response theory (Rost & Walter, this volume, chap. 18). Modern methodological approaches for metrical variables include multilevel models (Hox & Maas, this volume, chap. 19) and models of structural equation modeling (Eid, Lis-chetzke, & Nussbeck, this volume, chap. 20).
Was this article helpful?