An interpretable machine learning model for predicting intensive care unit mortality risk in elderly patients with concomitant cardiogenic shock and sepsis: Development and external validation using two large critical care databases
Han Chen, Yuming Wu, Pu Chen, Zijian Lin, Qingjuan Zheng, Ling WangIntroduction
Older adults with concomitant cardiogenic shock (CS) and sepsis represent a high-risk mixed-shock population in whom early prognosis is difficult. We aimed to develop and externally validate an interpretable machine-learning model for ICU mortality.
Methods
Patients aged ≥65 years with concomitant CS and sepsis were identified from MIMIC-IV v3.1 (n=1,219) and eICU-CRD v2.0 (n=166). MIMIC-IV was stratified 70:30 into training (n=853) and internal validation (n=366), while eICU-CRD was reserved for external validation. Feature selection used the intersection of Boruta and LASSO in the training cohort. Ten models were tuned by stratified 5-fold cross-validation; the primary model family was selected by the highest mean training cross-validated AUROC before validation. Performance assessment included bootstrap 95% confidence intervals, calibration, decision curve analysis, and SHAP.
Results
Nine predictors were retained: AKI, T2DM, respiratory rate, lactate, creatinine, BUN, ALT, SOFA, and SAPS II. Stacking ranked first by the training-CV model-selection rule (mean CV AUROC=0.690). AUROC was 0.742 (95% CI 0.690-0.791) in internal validation and 0.633 (0.546-0.718) in external validation. At the locked threshold, external sensitivity was 0.750 and specificity 0.441. The external Brier score of 0.287 exceeded the prevalence-only null Brier score of 0.237 (scaled Brier=-0.212). SOFA alone showed similar external discrimination (AUROC=0.646) with a lower Brier score (0.226), indicating no clear external advantage of the ensemble over simpler benchmarks. Cross-database standardized mean differences demonstrated substantial distribution shift.
Conclusions
The training-selected Stacking model showed moderate internal discrimination but limited external discrimination and probabilistic calibration, without a clear external advantage over simpler benchmarks. The model should be considered investigational pending prospective evaluation and site-specific validation or recalibration.