Critical Multiplism (CM) is closely related in its premises and goals with all previously presented milestones. In comparison to these, CM is more general (compared with the MTMM framework), broader in substantive scope (compared with Brunswik), and less technical (compared with covariance structure modeling and Generalizability Theory). Critical Multiplism is a way of thinking, a philosophy of science (Cook, 1985; Houts et al., 1986; Shadish, 1995). It starts from the premise that no perfect route to scientific knowledge exists and that all scientific options have their own strengths and weaknesses. Scientific options include theories, research designs, sampling strategies, measurement instruments, assessment procedures, rules for weighing and combining information, sta tistical models for analyzing data, guidelines for interpreting results, and principles for transforming scientific evidence into decisions and actions (e.g., intervention programs). Assuming that alternative research strategies always differ in their advantages and disadvantages, CM requires that research programs never rely on a single strategy but always combine several strategies. It is critical from the CM view that strategies are not chosen and combined at random but instead selected according to the principles of best quality and maximum heterogeneity. Heterogeneous strategies are preferable compared to homogeneous strategies because the convergence of results across highly dissimilar strategies is more convincing and increases the trustworthiness of evidence more than convergence among highly similar strategies. A specific application of this rule was outlined earlier: Combining heterogeneous assessment methods means that they share only the diagnostically relevant factors. The quality criterion is more difficult to operationalize. According to CM, high quality research requires that researchers make their implicit assumptions explicit, justify each component of their work (theory, design, sampling strategies, measurement methods, etc.), and invite members of the scientific community to challenge these justifications. Diversity in theory and method is considered in CM as the best safeguard against systematic error. Just like discrimination is a fundamental principle of knowledge, diversity is a fundamental prerequisite to determine convergence of evidence. Not surprisingly, CM supports multimethod assessment on the basis of quality and heterogeneity.
LOOKING AHEAD: SOME EMERGING ISSUES AND CHALLENGES
In what direction should multimethod work progress? All of this handbook's contributors likely have their own view regarding where progress is necessary and possible. Below, I address two important yet unresolved issues.
Was this article helpful?