Admission-based digital risk stratification for in-hospital mortality in COVID-19 during the vaccination era: Development and external validation of a machine learning model
Mónica Queipo, Jorge Mateo, Ana María Torres, Pilar Blasco-Segura, Julia BarbadoBackground
Later COVID-19 waves occurred after vaccine rollout, but documented vaccination and previous infection among hospitalized patients remained heterogeneous. Admission-based models using routine data may support risk stratification across epidemiological contexts when baseline-risk differences are considered.
Objective
To develop and externally validate a machine-learning model for early prediction of in-hospital mortality among patients hospitalized with COVID-19 during the vaccination era and to assess a parsimonious alternative.
Methods
This retrospective cohort study used electronic health record data from 244 adults admitted with SARS-CoV-2 pneumonia to Río Hortega University Hospital, Spain, between June 2021 and March 2022. Algorithms were evaluated using a common preprocessing pipeline, repeated stratified internal validation, Bayesian hyperparameter optimization, and training-only SMOTE. A final locked model was externally validated in 200 adults admitted during the first pandemic wave. A clinically guided five-variable XGBoost model was additionally developed and compared with the full 58-predictor model.
Results
Full XGBoost achieved ROC-AUCs of 0.94 internally and 0.92 externally, with corresponding PR-AUCs of 0.84 and 0.82. The five-variable model retained most discrimination, with internal and external ROC-AUCs of 0.92 and 0.90 and PR-AUCs of 0.81 and 0.78. In external validation, it achieved 88.3% mortality sensitivity and 90.7% specificity. External Brier scores were 0.11 for the full model and 0.12 for the parsimonious model.
Conclusions
The full model identified a broad admission-based prognostic signal, while the five-variable model retained most predictive performance with substantially lower input dimensionality. The parsimonious model is a more interpretable candidate for prospective evaluation; multicenter validation, local recalibration, and clinical-utility assessment remain required before implementation.