All Measures Have Shortcomings

Part of the motivation to use multiple methods comes when a researcher realizes that his or her measures are imperfect. At times researchers do not think about their measures, but simply take them on faith in an unquestioning way because they have face validity and other researchers use the same methods. At other times scientists use only one method because they adopt an ideology that asserts that a particular methodology is superior to all the others. Both are shortsighted beliefs because they fail to recognize that each and every type of measure has limitations.

Certain ideologies claim that measures within their tradition are superior to other types of measures, which are flawed. For example, many behav-iorists claimed that behavioral observation was the only method for understanding psychological phenomena, and other types of measures such as self-reports, biological assays, or projective techniques were badly flawed. Similarly, some researchers with a reductionistic worldview claim that biological measures, for example brain imaging and measures of hormones, are the only measures that are likely to advance science. These views are dangerous for several reasons. First, they fail to recognize that the biological and behavioral methods, although they might be more objective in some ways, have limitations and shortcomings. Measures of hormones, for example, can be influenced by extraneous factors such as time of day and the medications a participant is ingesting, and often the hormones do not map directly onto psychological constructs. Furthermore, the biological measures are given meaning in reference to other measures, such as self-reports or behavioral observations. When one delves into any biological measure, it becomes evident that the meaning of the measure can be complex and can be confounded by a host of artifacts.

Some scientists believe that only observable physical behavior should be studied. But behavioral observations can be contaminated by reactivity and by the perceptions of the coders. Furthermore, the behavior being coded will represent only one example of the construct in question and is likely to be influenced by factors other than the underlying construct. Thus, even though biological and behavioral observation measures can be valuable in our understanding, and certainly ought to be included in more investigations, these measures have flaws just as do self-reports, informant reports, and other "softer" measures.

Was this article helpful?

0 0

Post a comment