Prediction of cardiac arrest in prehospital patients using machine learning
J O Simonsen, J C Christensen, D M Modin, F G Gnesin, J T H Tfelt-Hansen, C T P Torp-Pedersen, K K Kragholm, J B Baech, H C Christiansen, C P Polcwiartek, C G Graff, M P A Porsborg, A S Sigurdsson, S R Rasmussen, T B S Biering-SoerensenAbstract
Background
Currently, no risk score model for the prediction of imminent cardiac arrest in a prehospital setting exists. Developing such a model might enable early intervention and improve survival.
Purpose
To develop a machine learning (ML) model for the prediction of cardiac arrest in a prehospital setting in patients presenting with chest pain or dyspnoea.
Methods
The study included all prehospital patients in Denmark from 2017 to 2021 presenting with chest pain or dyspnoea. Subjects from the East Denmark were used for training, and patients from West Denmark were used as geographical external validation. Overlapping patients were excluded. Recurrent patients were included. A case of cardiac arrest was defined as in-hospital cardiac arrest as the primary admission diagnosis, or out-of-hospital cardiac arrest witnessed by the emergency medical services and not witnessed by bystanders.Within. Patients experiencing cardiac arrest before predictors were collected were excluded. Two ML models (both XGBoost) were developed. Model 1 included 7 ambulance monitor measurements, and Model 2 further included 5 ECG-derived variables and 9 clinical assessment measures from the ambulance. Performance was assessed with ROC-AUC, PR-AUC, sensitivity, specificity, PPV, calibration plots, and DeLong statistics.
Results
The training set included 49,362 patients (62,812 prehospital cases). Mean age was 68 years (IQR 53:79), and 56.6% were male. The external validation set included 15,004 patients (18,812 prehospital cases). Mean age was 69 years (IQR 54:79), and 56.2% were male. In total, 379 (0.77%) patients in the training set, and 111 (0.74%) in the external validation set experienced cardiac arrest.
In external validation, Model 2, including all 21 parameters, resulted in a ROC-AUC of 0.868 [0.830:0.886] (Figure 1B) and a PR-AUC of 0.152 (0.109:0.223). The optimal top-left ROC threshold yielded a sensitivity of 73.91 (69.62:83.10), a specificity of 84.32 (69.01:86.38), and a PPV of 2.82% (2.05:4.07), corresponding to ~35 false positives per detected cardiac arrest. For a fixed sensitivity of 40.00%, the specificity was 97.56 (96.21:98.35), and the PPV was 9.16%(7.24:15.89), corresponding to ~10 false positives per detected cardiac arrest. The ROC-AUC was significantly higher in Model 2 compared to Model 1, including only monitor measures 0.797 [0.763:0.830], p<0.001. The predicted risk estimate showed good calibration with the observed event rates (Figure 1A). The three most important predictors of Model 2 were end-tidal CO2, lead-specific ST-elevations, and pain (VAS score)(Figure2).
Conclusion
We present a ML model for the prediction of imminent cardiac arrest in prehospital patients presenting with chest pain or dyspnoea. For a sensitivity of 40%, PPV was 9.16%, corresponding to ~10 false positives per detected cardiac arrest. The most important predictors were end-tidal CO2 and lead-specific ST-elevations.