Three Webbased Assessment Methods

In this section, three specific Web-based methods are presented: log file analysis as an example of a nonreactive method, using the randomized response technique in surveys conducted on the Web, and games as a cover format for Internet-based experiments.

Log file analysis is at the core of many nonreactive methods of behavioral research on the Web. Navigation behavior in Web sites can be captured as so-called click streams, both on an individual and on a group level. Scientific applications for Web log analysis can be used to extract information about behaviors from log files, calculate response times and nonresponse behavior, and find relevant differences between users' navigation behaviors. The tool STRATDYN (Berendt, 2002; Berendt & Brenstein, 2001), for instance, provides classification and visualization of movement sequences in Web navigation and tests differences between navigation patterns in hypertexts. Scientific LogAnalyzer (Reips & Stieger, 2004) is geared toward analyzing data provided on forms and was developed for the analysis of data from most types of Internet-based experimenting (see Using Web-Based Methods: An Example, this chapter, for a description of how to use Scientific LogAnalyzer). LOGPAT (Richter, Naumann, & Noller, 2003) is useful in analyzing sequential measures, (i.e., counting the frequency of specific paths or path types in a log file). Like Scientific LogAnalyzer, LOGPAT was developed as a platform-independent, Web-based tool. In addition to these scientific applications, a large number of commercial and free log file analysis programs are available that primarily focus on helping the user maintain a Web site. This type of software can help identify access errors, points of entry, and user paths through a Web site. Many of the applications are user friendly and create visually appealing graphical output. Example programs are Analog (http ://www. analog, cx/), FunnelWeb (, TrafficReport (, and Summary (

Testing large numbers of participants very quickly via the Web is particularly important for the success of research projects that depend on the availability of a large sample. Therefore, a Web-based format is always a good choice if the randomized response technique (RRT; Warner, 1965) is to be used. Researchers have demonstrated the feasibility of the RRT in a large number of studies (e.g., Antonak & Livneh, 1995; for an explanation of the method see Erdfelder & Müsch, this volume, chap. 15).

One of the better versions of the RRT, the cheater detection model by Clark and Desharnais (1998), which operates with an experimental between-subjects manipulation, has been repeatedly used on the Web (Müsch, Bröder, & Klauer, 2001; Reips & Müsch, 1999). Figure 6.1 shows a screen capture taken from the Web-based RRT study by Reips and Müsch on the feasibility and trustworthiness of a computerized random generator. The participant is asked to click on the random wheel on the left side of the window. A click results in one of two events: If the left portion of the window turns blue then a true answer to the question is requested. If the window turns red, then the participant is asked to answer with "Yes," independently of the true answer. This condition is compared with one in which a different "random" device independent of computers and the Internet is used: the participant's month of birth. From various other conditions the behavior's incidence rate and the proportion of "cheaters" (sic!) in the sample can be calculated, as well as the influence of the computerized "random wheel." The enhanced anonymity often associated with Web-based questioning has provided additional advantages when conducting RRT surveys on the Internet.

Web experiments designed in game style are likely to attract a very large number of participants who will participate with high motivation (e.g., Reips & Mürner, 2004; Ruppertsberg, Givaty, Van Veen, & Bülthoff, 2001). Ruppertsberg et al. (2001) used games written in Java as research tools for visual perception over the Internet. They concluded that presenting games "... on the Internet resulted in large quantities of useful data, and allowed us to draw conclusions about mechanisms in face recognition in a broader, less selected participant population" (p. 157).

dick wheel!

the rotating

Do you agree with the following

O Yes C No

FIGURE 6.1. A Web-based survey using the randomized response technique (RRT) in a study on the trustworthiness of computerized random generators.

Reips and Murner (2005) recently developed a Web site that allows researchers and students to develop their own Web-based Stroop experiments in an arcade game style. This Web site is available at The researcher can configure many aspects of the Stroop paradigm, like colors and names of objects, rules for events, rates for the different event types, speed, and the overall style in which the game is presented (i.e., "skins"). Access to the created Web experiment can be restricted using a login and password. The Web experiment is immediately available online, and the resulting data can be downloaded as tab-delimited text file in a format optimized for analysis in Scientific LogAnalyzer. Figure 6.2 shows the game pad page of "Stroop Invaders."

Using Web-Based Methods: An Example

Reading about an assessment method can be useful. However, to gain insights on a deeper level and to take concrete steps in acquiring knowledge about the method, it may be even more useful to experience it. Therefore, this section provides the opportunity to create and conduct a Web experiment, in a step-by-step fashion. Along the way, several useful tools for Web-based methods are presented, that is, WEXTOR (Reips & Neuhaus, 2002), the web experiment list (Reips & Lengler, 2005), the Web Experimental Psychology Lab (Reips, 2001), and Scientific LogAnalyzer (Reips & Stieger, 2004). A portion of McKenzie and Nelson's (2003) "cup experiment" is recreated for replication on the Web. This study deals with the information implic-

FIGURE 6.2. "Stroop Invaders": A Web site that allows researchers, teachers, and students to design and conduct Web-based Stroop experiments.

itly conveyed by the speaker's choice of a frame— for instance, describing a cup as being "half full" or "half empty."

Was this article helpful?

0 0

Post a comment