PP500 Topic: AS22–Quality and Safety/Errors/Data Management/Other: DEVELOPMENT OF ARTIFICIAL NEURAL NETWORK MODELS FOR PAEDIATRIC CRITICAL ILLNESS IN SOUTH AFRICA

Abstract

Aims & Objectives: Failures in identification, resuscitation and appropriate referral have been identified as significant contributors to avoidable severity of illness and mortality in South African children. In this study, artificial neural network models were developed to predict a composite outcome of death before discharge from hospital or admission to the PICU. These models were compared to logistic regression and XGBoost models developed on the same data in cross-validation. Methods: Design: Prospective, analytical cohort study. Setting: A single centre tertiary hospital in South Africa providing acute paediatric services. Patients: Children, under the age of 13 years presenting to the Paediatric Referral Area for acute consultations. Outcomes: Predictive models for a composite outcome of death before discharge from hospital or admission to the PICU. Modelling: Nine candidate predicitive models were developed in the Jupyter Notebooks environment using Python3. Model performance was validated by stratified cross-validation. Results: 765 patients were included in the data set with 116 instances (15.2%) of the study outcome. All developed models demonstrated discrimination with mean ROC AUCs greater than 0.8 and mean PRC AUCs greater than 0.53. ANN1 demonstrated the best discrimination with a ROC AUC of 0.84 and a PRC AUC of 0.64 Model calibration was variable, with most models demonstrating weak calibration. Decision curve analysis demonstrated that all models were superior to baseline strategies, with ANN1 demonstrating the highest net benefit. Conclusions: All models demonstrated satisfactory performance, with the best performing being an ANN model. Parsimonious models demonstrated strong performance and should be considered, given their ease of implementation in practice. Keywords: Critical Illness, Triage, Machine Learning, Children