To evaluate the value of surveillance data for detecting disease in a single patient, an evaluator employs methods identical to those described in Chapter 20.
The evaluator assembles a set of individuals known to have the disease (termed cases) and a set of individuals without the disease to serve as controls. The cases can be established by reviewing medical records, veterinarian records, or lab results. The evaluator uses a case-detection method that is appropriate to the data and computes the sensitivity and specificity of detection and the area under the receiver operator characteristic (ROC) curve, as described in Chapter 20. The evaluator uses the area under the ROC curve as a measure of the data's informational value for detecting cases of the disease in question. Since the area under the ROC curve is not an absolute measure of value, these evaluations typically compare the area under the ROC curve with that obtained from an identical analysis using some other type of surveillance data. Alternatively, evaluators rely on knowledge about what levels of sensitivity and specificity are required by a biosurveillance system. Some example published studies are (Chapman et al., 2003b, Espino and Wagner, 2001, Ivanov et al., 2003).
If the evaluation fails to show that the data can discriminate between patients with the disease and without the disease (i.e., the area under the ROC curve equals 0.5) or that the performance is not sufficient for a particular application, the evaluator should question whether a different case-detection algorithm might produce better results. Even if discrimination ability is found, an evaluator may experiment with different case-detection algorithms to establish the best possible performance that can be achieved with the data.
Was this article helpful?