Severity Scores in Emergency Department Patients with Presumed Infection: A Prospective Validation Study

The objectives of this study were to 1) validate a number of severity of illness scores in a large cohort of emergency department patients admitted with presumed infection and 2) compare the performance of scores in patient subgroups with increasing mortality: infection without systemic inflammatory response syndrome, sepsis, severe sepsis, and septic shock. Design: Prospective, observational study. Setting: Adult emergency department in a metropolitan tertiary, university-affiliated hospital. Patients: Emergency department patients admitted with presumed infection. Interventions: None. Methods: Consecutive emergency department patients admitted with presumed infection were identified over 160 weeks in two periods between 2007 and 2011. Clinical and laboratory data sufficient to calculate Mortality in Emergency Department Sepsis score, Acute Physiology and Chronic Health Evaluation II, Simplified Acute Physiology Score II, Sequential Organ Failure Assessment, and the Severe Sepsis Score were entered into a database. Model discrimination was quantified using area under the receiver operating curve. Calibration was assessed using visual plots, Hosmer-Lemeshow statistics, and linear regressions of observed and predicted values. Measurements and Main Results: A total of 8,871 patients were enrolled with 30-day mortality of 3.7%. Area under the receiver operating curve values for the entire cohort were: Mortality in Emergency Department Sepsis score of 0.92, Simplified Acute Physiology Score II and Acute Physiology and Chronic Health Evaluation II scores of 0.90, Sequential Organ Failure Assessment score of 0.86, and Severe Sepsis Score of 0.82. Discrimination decreased in subgroups with greater mortality for each score. All scores overestimated mortality, but closest concordance between predicted and observed mortality was seen with Mortality in Emergency Department Sepsis score. Conclusions: The decrease in area under the receiver operating curve seen in subgroups with increasing mortality may explain some variation in results seen in previous validation studies. Scores developed in intensive care settings overestimated mortality in the emergency department. Our results underscore the importance of employing predictive models developed in similar patient populations. The Mortality in Emergency Department Sepsis score outperformed more complex predictive models and would be the most appropriate scoring system for use in similar emergency department populations with a wide spectrum of mortality risk. © 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc.