External validation of artificial intelligence–based electrocardiographic models for heart failure detection across diverse clinical settings
A Champetier, L Chollet, Klara Rumora, I Strebel, J Boeddinghaus, C Puelacher, T Stolte, K Durak, J Simonavicius, F Mahfoud, M J Zellweger, T Nestelberger, O Pfister, C Mueller, D GualandroAbstract
Introduction
In up to 50% of patients, heart failure (HF) is first diagnosed at the time of hospitalization for acute HF, representing a missed opportunity for earlier treatment. Artificial intelligence–enhanced electrocardiography (AI-ECG) offers a rapid and scalable opportunity for HF detection and risk stratification. Two recently published AI-ECG models were developed using distinct proxy outcomes for HF: DeepECG, a Canadian model trained to detect left ventricular ejection fraction <40% as a surrogate for heart failure with reduced ejection fraction (HFrEF), and ECG2HF, a US model trained to predict incident HF using longitudinal electronic health record data. However, their performance for detecting heart failure in a pooled European cohort derived from five prospective studies remains unknown.
Methods
In this pooled external validation, we analyzed digital 12-lead ECGs from five prospective studies conducted at a single European center across diverse clinical settings, including outpatient diagnostic populations, perioperative inpatients at high cardiovascular risk, cardiac rehabilitation patients, and a national cohort undergoing transcatheter aortic valve implantation. HF was centrally adjudicated by two independent cardiologists according to guideline-recommended criteria, incorporating symptoms, natriuretic peptide levels with clinical adjustments, and structural and/or functional cardiac abnormalities. Model performance was assessed using discrimination and precision–recall analysis. The primary outcome was HF, and the secondary outcome was HFrEF.
Findings
Among 7124 patients with curated diagnoses (median age 70 years [IQR 61–78], 2279 [32%] women), 3153 (44%) had HF, including HFpEF in 1564 (21%), HFmrEF in 537 (7%), and HFrEF in 633 (9%). For the detection of HF, DeepECG and ECG2HF showed similar discrimination (AUC 0.72 [95% CI 0.71–0.73] vs 0.73 [0.71–0.74]) and comparable precision–recall performance (AUPRC 0.69 vs 0.68). For HFrEF, DeepECG demonstrated superior discrimination compared with ECG2HF (AUC 0.90 [0.89–0.91] vs 0.85 [0.84–0.87]) and higher precision–recall performance (AUPRC 0.52 vs 0.35). The combination of both models consistently improved performance, yielding higher discrimination and precision–recall metrics for HF (AUC 0.74, AUPRC 0.71) and HFrEF (AUC 0.91, AUPRC 0.54).
Interpretation
In this large European cohort spanning five clinical scenarios of heart failure detection, the performance of the deep learning models DeepECG and ECG2HF, which were developed to identify proxies of heart failure, was overall moderate. However, the improved performance observed with their combination suggests that each model captures distinct and complementary features related to heart failure. This highlights the potential limitations of relying on surrogate endpoints and supports the development of models trained directly on heart failure outcomes to improve risk stratification.AUC and AUPRC for HFAUC and AUPRC for HFrEF