Evaluation of the SMS-ICU score for multicontinental use: A federated external validation study using critical care registry data

Abstract

Objective: To evaluate the suitability of the Simplified Mortality Score for the Intensive Care Unit (SMS-ICU) for use in critical care registries across multiple continents. Design: A federated external validation study. Setting: Intensive care units (ICUs) belonging to 12 critical care registries on five continents. Participants: Individuals over the age of 18 years who were admitted to participating ICUs during the 2023 calendar year. Readmissions within the same hospital encounter and interhospital transfers were excluded. Interventions: None. Main outcome measures: The primary outcome was all-cause in-hospital mortality. A secondary analysis considered ICU mortality. Results: 12 registries participated (three from Africa, four from Asia, one from Oceania, two from Europe and two from South America). In total, 494,705 admissions were included from 546 ICUs in 13 countries. Seven registries systematically reported hospital outcomes and were included in the primary analysis. Discrimination was acceptable (pooled area under the receiver operating characteristic curve [AUROC] 0.76 [95% CI 0.72-0.80]). Calibration and overall model fit varied between registries (Brier scores 0.09-0.14). In the secondary analysis of ICU outcomes in all 12 registries, discrimination remained acceptable but with higher variation between registries (pooled AUROC 0.73 [0.69-0.77]). Calibration and overall model fit also varied widely (Brier scores 0.07-0.46). Conclusions: SMS-ICU may be used as an index of illness severity in international intensive care populations. This may facilitate international collaborative research and synthesis of findings in systematic reviews and meta-analyses.