Enhancing diabetes prediction through probability-based correction: a methodological approach

Abstract

Predictive healthcare analytics demands accurate predictions from interpretable models for early diagnosis and intervention on diabetes prognosis, which remains a well-established challenge. This study presents a new probability-based correction method to enhance the performance of a model in diabetes prediction. Initial model comparisons are performed using the PyCaret framework to identify the baseline model. Logistic regression was selected due to its simplicity, interpretability, and its higher accuracy, which outperformed other models. To further facilitate future research in this field, this study was conducted using a noisy dataset without any changes or preprocessing steps other than those available in the dataset from the producer. This intentional decision meant that the new probability-based method could be evaluated in isolation without any additional modifications being applied. The proposed correction method adjusts predictions into borderline probability intervals to obtain more accurate classifications. This approach increased the model accuracy by 6% from 75% to 81%, thus proving successful in resolving the misclassification problem with higher risk. This approach outperforms state-of-the-art methods and demonstrates its generalizability in enhancing the certainty of downstream clinical decisions.