aFrom Espino, J., Wagner, M. (2001a). The accuracy of ICD-9 coded chief complaints for detection of acute respiratory illness. In: Proceedings of American Medical Informatics Association Symposium, 164-8, with permission.
bFrom Betancourt, J. A. (2003). An evaluation of the Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE). Doctoral dissertation, George Washington University, with permission.
cFrom Beitel, A. J., Olson, K. L., Reis, B. Y., et al. (2004). Use of emergency department chief complaint and diagnostic codes for identifying respiratory illness in a pediatric population. Pediatr Emerg Care 20:355-60, with permission.
dFrom Ivanov, O., Wagner, M. M., Chapman, W. W., et al. (2002). Accuracy of three classifiers of acute gastrointestinal syndrome for syndromic surveillance. In:
Proceedings of American Medical Informatics Association Symposium, 345-9, with permission.
eFrom Guevara, R. E., Butler, J. C., Marston, B. J., et al. (1999). Accuracy of ICD-9-CM codes in detecting community-acquired pneumococcal pneumonia for incidence and vaccine efficacy studies. Am J Epidemiol 149:282-9, with permission.
^From San Gabriel, P., Saiman, L., Kaye, K., et al. (2003). Completeness of pediatric TB reporting in New York City. Public Health Rep 118:144-53, with permission. gFrom Rosenblum, L., Buehler, J. W., Morgan, M. W., et al. (1993). HIV infection in hospitalized patients and Medicaid enrollees: the accuracy of medical record coding. Am J Public Health 83:1457-9, with permission.
respiratory syndrome, respectively, and for their lower respiratory ICD code set was 0.87 and 0.99 for lower respiratory syndrome, respectively.
Notably, Beitel and colleagues studied the combined information present in both chief complaints and ICD codes. They assigned patients to respiratory if they had either a respiratory chief complaint or an ICD code in the respiratory code set. The sensitivity and specificity of the combined definition for respiratory were 0.72 and 0.97, respectively. These results vary only slightly from the sensitivity and specificity of the respiratory ICD code set alone.
Detection of More Diagnostically Precise Syndromes. Several studies have measured the case detection accuracy of ICD codes available only after a patient has left the hospital. We discuss three examples of such studies here.
Guevara et al. (1999) studied the sensitivity and specificity of ICD codes and code sets for the detection of the disease pneumococcal pneumonia. The ICD codes were part of data collected during a study of community-acquired pneumonia. They used the results of microbiology cultures as a gold standard. They found that ICD code 481 had the highest sensitivity—58.3%—for pneumococcal pneumonia (out of all six ICD codes studied) and a specificity of 97.5% (Table 23.13). Of the six ICD code sets they studied, an ICD code set with four ICD codes had the highest sensitivity and specificity-81.2% and 96.0%, respectively. One note of caution in interpreting these results is they limited the study to patients who met the inclusion criteria for community-acquired pneumonia. Thus the sensitivity and specificity results reported are specifically for discriminating among pneumococcal and other causes of community-acquired pneumonia in hospital patients.
San Gabriel et al. (2003) studied the accuracy of hospital discharge ICD codes for the detection tuberculosis in children. The gold standard against which they compared ICD codes was medical record review. An ICD code set that contained all ICD codes with "tuberculosis'' in their definition had a sensitivity and specificity of 57% and 75%, respectively, for the detection of tuberculosis in children.
Rosenblum et al. (1993) studied the accuracy of hospital discharge ICD codes for the detection of human immunovirus (HIV) infection and acquired immunodeficiency syndrome (AIDS) (Rosenblum et al., 1993).They found that the sensitivity and specificity of an ICD code set (that contained four ICD codes for HIV infection and AIDS) for HIV was 98% and 89%,12 respectively. Similarly, the sensitivity and specificity of the ICD code set for AIDS were 97% and 58%, respectively.13
Summary of Studies of Case Detection from ICD Codes. With respect to Hypothesis 1, the literature on accuracy of both disease-detection and syndrome-detection from ICD code analysis suggest that:
1. ICD-coded diagnoses and chief complaints contain information about syndromic presentations and diagnoses of patients and existing code sets can extract that information.
2. For syndromes that are at the level of diagnostic precision of respiratory or gastrointestinal, it is possible to automatically classify ED and office patients (both pediatric and adult) from ICD codes with a sensitivity of approximately 0.65 and a specificity of approximately 0.95. This level of accuracy is similar to that achievable with chief complaints.
3. For diagnoses that are at the level of diagnostic precision of disease (e.g., pneumonia) or disease and organism (e.g., pneumococcal pneumonia) it is possible to automatically classify patients from hospital discharge data.
4. Sensitivity of classification is better for some syndromes and diagnoses than for others.
5. The specificity of case detection from ICD codes and code sets is less than 100%, meaning that daily aggregate counts will have false positives among them due to falsely classified patients.
Was this article helpful?