Harnessing hybrid stacking ensemble learning for accurate pulmonary embolism diagnosis using tabular clinical data

Abstract

Pulmonary Embolism (PE) is a serious condition that can be life-threatening if not diagnosed promptly as timely clinical decisions are critical. This research aims to present a hybrid stacking ensemble (HSE) framework for PE prediction using tabular clinical data from the RSNA-STR-PE dataset. The proposed ensemble combines four complementary base learners: (SAINT transformer, XGBoost, LightGBM, and MLP) to capture diverse feature representations and decision patterns. To optimize ensemble performance, the probabilistic outputs of the base models are weighted using the Marine Predators Algorithm (MPA), enabling adaptive learning of optimal combination weights. These optimized predictions are subsequently fused through a logistic regression meta-learner with L2 regularization, enhancing generalization while mitigating overfitting. Experimental results demonstrate that the proposed MPA-optimized-HSE achieves an accuracy of 92.3% and 0.91 for AUROC, outperforming all individual base models. Furthermore, the MPA-optimized-HSE provides a robust and interpretable approach for improving PE prediction from tabular clinical data, highlighting its potential utility in real-world diagnostic settings.