Our evaluation criteria examines algorithm performance on an AMOC curve (Fawcett and Provost, 1997). AMOC curves are described in more detail in Chapter 20. These results were generated from 100 synthetic data sets described in the WSARE paper (Wong et al., 2003), and are available at www.cs.cmu.edu/ ~awm/wsare-data. The AMOC curve in Figure 15.6 plots detection time versus false positives over alarm thresholds ranging from 0 to 0.2 in 0.001 increments. The lower alarm thresholds yield lower false positives and higher detection times while the converse is true with higher alarm thresholds. The best possible detection time is one day, as shown by the dotted line at the bottom of the graph. We add a one-day delay to all detection times to simulate reality where current data is only available after a 24-hour delay. Any alert occurring before the start of the simulated anthrax attack is treated as a false positive. Detection time is calculated as the first alert raised after the release date. If no alerts are raised after the release, the detection time is set to 14 days.
The Israel Center for Disease Control evaluated WSARE 3.0 retrospectively using an unusual outbreak of influenza type B that occurred in an elementary school in central Israel (Kaufman et al., 2004). WSARE 3.0 was applied to patient visits to community clinics between the dates of May 24,2004 to June 11, 2004. The attributes in this data set included the visit date, area code, ICD-9 code, age category, and day of week. The day of week was used as the only environmental attribute. WSARE 3.0 reported two rules with p-values at 0.002 and five other rules with p-values below 0.0001. Two of the five anomalous patterns with p-values below 0.0001 corresponded to the influenza outbreak in the data. The rules that characterized the two anomalous patterns consisted of the same three attributes of ICD-9 code, area code and age category, indicating that an anomalous pattern was found involving children aged 6 to 14 having viral symptoms within a specific geographic area.WSARE 3.0 successfully detected the outbreak on the second day from its onset.
Was this article helpful?