Multimodal ECG–language contrastive pretraining enables detection of structural heart disease from 12-Lead ECG
A El-Medany, J Huo, K Patlatzoglou, H Aggour, G Khattak, N Pai, Z Rossaye, A Birdi, B Zeidaabadi, L Pastika, J Barker, N Peters, A Sau, S Khan, F S NgAbstract
Background
The 12-lead electrocardiogram (ECG) is inexpensive and widely accessible, yet may encode latent information relating to structural heart disease (SHD). [1] Recent advances in multimodal representation learning, particularly ECG-language contrastive pretraining on paired ECG-report datasets, offer a scalable strategy to improve downstream diagnostic performance. [2] However, the clinical utility and external generalisability of such models for SHD detection remain incompletely validated.
Purpose
To develop and externally validate a multimodal ECG-language contrastive learning model for the detection of structural heart abnormalities, including reduced left ventricular ejection fraction (LVEF <45%), increased left ventricular wall thickness (LVWT >1.3 cm), and moderate-or-greater valvular heart disease.
Methods
A multimodal ECG-language model was pretrained using contrastive learning on paired 12-lead ECGs and clinical reports from the Harvard-Emory ECG Database. The ECG encoder utilised a 1D ResNet-34 architecture, and the text encoder employed MedCPT. The model was subsequently fine-tuned using the EchoNext Mini training dataset (n = 72,475 ECGs). Internal evaluation was performed on the EchoNext Mini test set (n = 5,442 ECGs). External validation was conducted on the Beth Israel Deaconess Medical Center (BIDMC) dataset (n = 61,520 ECGs) and an independently collected cohort (n = 39 ECGs). Fine-tuning tasks included detection of: LVEF <45%, LVWT >1.3 cm, and moderate-or-greater aortic stenosis (AS), aortic regurgitation (AR), mitral regurgitation (MR), and tricuspid regurgitation (TR). Model performance was evaluated using area under the receiver operating characteristic curve (AUC).
Results
On the EchoNext Mini internal test set, the model achieved AUCs of 0.90 (LVEF <45%), 0.77 (LVWT >1.3 cm), 0.85 (AS), 0.76 (AR), 0.83 (MR), and 0.84 (TR). On the external BIDMC dataset, corresponding AUCs were 0.86, 0.71, 0.73, 0.63, 0.76, and 0.78. On the independently collected dataset, AUCs were 0.95, 0.77, 0.34, 0.56, 0.97, and 0.74, respectively. Performance remained strongest for reduced LVEF and mitral/tricuspid regurgitation across cohorts, with moderate degradation under cross-centre distribution shift.
Conclusion
An ECG-language contrastive pretraining framework enables accurate detection of SHD from 12-lead ECG signals. The model demonstrates robust internal performance and maintains reasonable discrimination in large external cohorts, supporting multimodal pretraining as a promising strategy to enhance ECG-based screening for SHD. Further large-scale prospective validation, particularly in routine clinical pathways, is warranted.