B80-3-13 Improving the Generalizability of Sleep-based Cardiovascular Risk Prediction Using Super Learner Ensemble

Rationale

In the context of Obstructive Sleep Apnea (OSA) and cardiovascular health, machine learning (ML) holds promise for risk prediction; however, models developed in epidemiologic cohorts often perform poorly when translated to clinical populations. This gap occurs because clinical populations differ in covariates and outcomes, and because ML models may learn spurious associations (e.g., cohort visit schedules) rather than stable invariant mechanisms (e.g., underlying disease biology). We hypothesize that a SuperLearner ensemble that debiases dropout and explicitly handles missing-not-at-random (MNAR) data will improve predictive robustness in new clinical populations. By accounting for distributional shift and informative missingness typical of real-world clinical settings, this approach aims to bridge the gap between epidemiologic model development and reliable clinical deployment.

Method

We analyze 2159 participants in the Multi-Ethnic Study of Atherosclerosis (MESA) sleep ancillary study with no prior cardiovascular events (CVE), using exam 5 clinical, sleep and coronary artery calcium (CAC) data, to predict time-to-CVE. We divide the dataset into training (65%) and test (35%) sets such that participants from sites in the Midwestern US are in the test set to simulate distributional shift of covariates and outcomes. By combining multiple prediction models, the SuperLearner is designed to prioritize signals that generalize across populations, and we benchmark its performance against conventional single-model approaches while adjusting for missing data and dropout.

Results

We evaluated time-independent risk discrimination using the concordance index (C-index). Cross-validated training results reflect in-distribution generalization, where the SuperLearner and the best individual model performed similarly and both outperformed standard survival models after accounting for missingness. Native handling of MNAR data performed comparably to iterative imputation under MAR assumptions. On held-out test data reflecting out-of-distribution generalization, which was our primary focus, the SuperLearner explicitly modeling MNAR achieved the highest statistically significant discrimination (C-index 0.716), outperforming gradient boosting with Cox loss under MAR (0.705) and SuperLearners assuming MCAR (0.706) or MAR (0.698). The SuperLearner also demonstrated lower variance than its strongest base learner.

Conclusion

In the MESA-SLEEP cohort, we show that the SuperLearner-based ML framework offers increased robustness and out-of-distribution generalization for time-independent risk prediction tasks compared to common survival models and a single model selection strategy. The results suggest that accounting informative missingness helps with out-of-distribution generalizability in comparison to common imputation methods. The SuperLearner framework shows promising results and should be evaluated for its reliability in translating epidemiological models to clinical settings.

This abstract is funded by: R01HL168897