We evaluated the performance of PANDA on data sets produced by injecting simulated ED cases into a background of actual ED cases obtained from several hospitals in Allegheny County. In accordance with (HIPAA) regulations, all personal identifying information was removed from these actual ED cases. The simulated cases of anthrax were produced by the BARD simulator (see Chapter 19) that models the effects of an airborne anthrax release using an independently developed
Gaussian plume model of atmospheric dispersion of anthrax spores (Hanna et al., 1982).
Given weather conditions and parameters for the location, height, and amount of the airborne anthrax release, the Gaussian plume model derives the concentration of anthrax spores that are estimated to exist in each zip code. The extent of an outbreak for a zip code is influenced by the spore concentration in the zip code and the number of people living in the zip code. The output from the simulator consists of a list of anthrax cases, where each case consists of a date-time field and a zip code. The full details of the model are in Chapter 19. For our experiments, we selected historical meteorological conditions (e.g., wind direction and speed) for Allegheny County from a random date as the meteorological input to the simulator. The height of the simulated release was sampled from a prior distribution, created using expert judgment (see Chapter 19). This distribution was skewed towards heights less than 1500 feet. Finally, the release locations were sampled from a prior distribution, which favors release locations that would be expected to infect large numbers of people given the current meteorological conditions.
The output of the BARD simulator cannot be used directly by PANDA because a full evidence vector for a case includes information about the patient's age and gender. As a result, we took the partially complete patient cases produced by the simulator and probabilistically assigned the age and gender fields using the person model Bayesian network. The age of the patient is sampled from the conditional distribution of age given the home zip code of the patient and given the fact that the patient had respiratory symptoms when admitted. We use a similar procedure for determining the gender.
The anthrax release simulator that we used generally generates multiple downwind cases of anthrax that span several zip codes. The simulator also includes a minimum incubation period of 24 hours after the release during which no cases of anthrax are generated.5 Beyond that minimum period, the incubation period varies, with greater airborne concentrations of anthrax leading to a shorter incubation period, in general, than lesser concentrations.
In order to evaluate the detection capability of PANDA, we generated data sets corresponding to simulated releases of anthrax of the following amounts: 1.0, 0.5, 0.125, and 0.015625.6 For each release amount, we created 96 data sets, each with a unique release location. For each month in 2002, we chose eight random release dates and times to use with the simulator, thus
5 Arguably this minimum could be larger. The results in this chapter, however, emphasize detection time that is relative to the incubation time, rather than detection time from the point of the simulated release. Thus, the exact choice of an incubation time is less critical.
6 The units of concentration are not reported here in order to avoid providing results that could pose some degree of security risk. The concentration of 0.015625 was included for technical reasons.
producing a total of 96 different anthrax release data sets. We used only 91 data sets for the 0.015625 concentration because five of the data sets generated had no reported anthrax cases. PANDA was applied to monitor the data from each data set, starting on midnight of January 4, 2001 and extending through to six days after the simulated anthrax release occurred.
We measured the performance of PANDA using an AMOC curve, introduced by (Fawcett 1999) and described in Chapter 20. It plots time to detection as a function of false alarms per week. The points on the AMOC curve are generated by determining the false-positive alarm rate (ranging from 0 to 1) and detection time of the algorithm over a range of alarm thresholds, where an alarm is considered to be raised if the posterior probability of an Anthrax Release = yes exceeds the given alarm threshold. Since no known releases of anthrax have ever occurred in the region under study, the false-positive alarm rate was measured by determining the fraction of monitored hours that the release probability exceeded the alarm threshold under consideration for the period starting on January 4, 2001 and continuing until the simulated release date for the particular data set. In order to measure the timeliness of detection, we counted the number of hours that passed between the time of the simulated anthrax release and the first time the posterior probability of Anthrax Release = yes (as produced by PANDA) exceeded the alarm threshold. If no alarms were raised within six days after the simulated release point, the detection time was set to be 144 hours.
Was this article helpful?