In the history of psychological assessment, the various computer-based test applications evolved differently. The relatively more routine tasks were initially implemented, and the applications of more complex tasks, such as interpretation, took several decades to become available.
The earliest computer-based applications of psychological tests involved scoring and data processing in research. Almost as soon as large mainframe computers became available for general use in the 1950s, researchers began to use them to process test development information. In the early days, data were input for scoring by key entry, paper tape, or cards. Today optical readers or scanners are used widely but not exclusively. It is also common to find procedures in which the respondent enters his or her responses directly into the machine using a keyboard. Allard, Butler, Faust, and Shea (1995) found that computer scoring was more reliable than manual scoring of test responses.
In the 1950s, some commercial services for scoring psychological tests for both research and clinical purposes emerged. These early services typically provided summary scores for the test protocols, and in some cases, they provided a profile graph with the appropriate levels of the scale elevation designated. The technology of computer graphics of the time did not allow for complex visual displays or graphing a profile by connecting the dots, and the practitioner needed to connect the dots manually to complete the profile.
As computer use became more widespread, its potential advantage to the process of profiling of scores and assigning meaning to significantly elevated scores came to be recognized. A research group at Mayo Clinic in Rochester, Minnesota developed a computer program that actually provided rudimentary interpretations for the Minnesota Multi-phasic Personality Inventory (MMPI) results of patients being seen at the hospital (Rome et al., 1962). The interpretive program was comprised of 110 statements or descriptions that were based on empirical correlates for particular MMPI scale elevations. The program simply listed out the most relevant statements for each client's profile. This system was in use for many years to assess psychopathology of patients undergoing medical examinations at Mayo Clinic.
In 1963 Piotrowski completed a very elaborate computer program for Rorschach interpretation (Exner, 1987). The program was based on his own interpretive logic and included hundreds of parameters and rules. Because the program was too advanced for the computer technology available at that time, Piotrowski's program never became very popular. However, it was a precursor of modern computer programs for calculating scores and indexes and generating interpretations of Rorschach data.
Evolution of More Complex Test Interpretation and Report Generation
It wasn't long until others saw the broader potential in computer-based test interpretation. Fowler (1969) developed a computer program for the drug company, Hoffman-La Roche Laboratories, that not only interpreted the important scales of the MMPI but also combined the interpretive
statements into a narrative report. Several other computer-based systems became available in the years that followed— for example, the Caldwell Report (Caldwell, 1996) and the Minnesota Report (Butcher, 1982).
Computer administration has been widely used as a means of obtaining response data from clients. This response format has many advantages over traditional manual processing methods—particularly the potential time savings, elimination of the possibility that respondents would make errors while filling out handwritten answer sheets, and elimination of the possibility that clinicians and technicians would make errors while hand-scoring items .
The flexibility of the computer offers the option of adapting the test to suit the needs and preferences of the test taker. The administration of test items in a paper-and-pencil inventory requires that the test taker respond to each and every question regardless of whether it applies. Psychologists have been interested in modifying the administration of test items to fit the respondent—that is, to tailor a test administration to be analogous to an interview. For example, in an interview, if a question such as Are you married? is answered no, then all subsequent questions take this response into account and are branched away from seeking responses to items pertinent to being married. In other words, the items are administered in an adapted or tailored manner for the specific test taker. The comparability and validity of this method (known as computerized adaptive testing) have been explored in several studies (e.g., Butcher, Keller, & Bacon, 1985). Roper, Ben-Porath, and Butcher (1995) examined an adaptive version of the MMPI-2. Five hundred and seventy-one undergraduate psychology students were administered three versions of the MMPI-2: a booklet version, an adaptive computerized version, and a conventional computerized version. Each participant took the same format twice, took the booklet and adaptive computerized versions (in counterbalanced order), or took the conventional and adaptive computerized versions (again, in counterbalanced order). There were few statistically significant differences in the resulting mean scale scores between the booklet and adaptive computerized formats.
Available computer interpretation systems, even the most sophisticated report-generating programs, are essentially look up, list out programs—that is, they provide canned interpretations that have been stored in the computer to be called up when various test scores and indexes are obtained. The computer does not actually make decisions but simply follows instructions
(often very complex and detailed ones) about the statements or paragraphs that are to be printed out. The use of computers to actually make decisions or simulate what the human brain does in making decisions—an activity that has been referred to as artificial intelligence—has not been fully accomplished in the assessment field. One program that comes closest to having the computer actually make the decisions is available in the Minnesota Personnel Screening Report (Butcher, 1995). In this system, the computer has been programmed with decision rules defining an array of test scores and decisions (e.g., manages stress well). The computer program determines the scores and indexes and then decides which of the summary variables are most appropriate for the range of scores obtained.
Butcher (1988) investigated the usefulness of this computer-based MMPI assessment strategy for screening in personnel settings. A group of 262 airline pilot applicants were evaluated by both expert clinicians and by computer-based decision rules. The overall level of adjustment of each applicant was rated by experts (using only an MMPI profile) on a Likert-type scale with three categories: adequate, problems possible, and problems likely. The computer-based decision rules were also used to make determinations about the applicants. Here, the categories of excellent, good, adequate, problems possible, and poor were used to classify the profiles. The results showed high agreement between the computer-based decisions and those made by clinicians in rating overall adjustment. Over 50% of individuals falling into the adequate category based on the computer-based rules were given ratings of adequate by the clinicians. There was agreement between the computer rules and clinician judgment on the possibility of problems being present in 26.7% of cases. Over 60% of individuals rated as poor by the computer rules were given problems likely ratings by the clinicians. This study indicated that there can be substantial agreement between clinicians and the computer when an objectively interpreted test is used. The study did not, however, provide information on the external validity of either approach because no criteria were available to allow for an assessment of the relative accuracy of either method.
Computer-based technological developments are advancing more rapidly than is the psychological technology to support psychological test usage on the Internet. The growth of the Internet and broadening commercial uses have increased the potential to administer, score, and interpret psychological tests online. Commercial test publishers have been receiving a great deal of pressure from test users to make more test-based services available on the Internet. The ethics of psychological test usage, standards of care, and the basic psychological test research have not kept up with the growth spurt of the Internet itself. Consequently, there are many unanswered questions as psychologists move into the twenty-first century with the almost limitless potential of test applications facing the field. Later in this chapter, we address a number of these issues.
Was this article helpful?