Assessing generalizability of an AI model for cardiac amyloidosis detection in ECGs: interim results of a multi-centre, external validation study
F S Ahmad, O Lairez, M Rienstra, P van der Meer, E Zorio, L Alvarez Acosta, R Lopez Garcia, I Sanchez Rodriguez, F De Guio, R Ruiz, J M Lillo Castellano, M Marina Breysse, E Gonzalez Lopez, P Garcia PaviaAbstract
Introduction
Misdiagnosis and delay in diagnosis are frequent in patients with transthyretin cardiac amyloidosis (ATTR-CM). Leveraging Artificial Intelligence (AI) and simple and accessible tests such as the electrocardiogram (ECG) have the potential to aid with ATTR-CM early recognition.
Purpose
An internal validation study was completed at a Spanish expert center to develop an AI-ECG model to identify patients with ATTR-CM from 12-lead ECGs. This is a preliminary assessment of the generalizability of the model in an actively recruiting international, multi-centre external validation study.
Methods
We analyzed all routine care 12-lead ECGs from two ongoing clinical studies included in the interim data cut-off of February 24th 2026. Confirmed investigator diagnosis (ATTR-CM cases), confirmed negative diagnosis (controls for study 1) or diagnosis absence (controls for study 2) per routine care were used as ground truth. All ECGs (ATTR-CM: 440 [2.86%]; controls: 14,968 [97.14%]) were previously unseen by the model and were used for external validation. The AI model, a Time-Slicing Convolutional Neural Network previously described1, was optimized to discriminate between ATTR-CM cases and controls. Raw ECG signals were preprocessed using standard noise-reduction filtering and per-lead normalization. Additionally, class weighting was applied to the loss function to mitigate class imbalance and Monte Carlo cross-validation was utilized to ensure performance robustness and mitigate overfitting. Performance metrics included the Area Under the Curve (AUC), Sensitivity, and Specificity. Finally, an exploratory analysis was conducted to assess external validation when using one of the study 1 datasets for model training while maintaining the remaining data as external and independent from model training.
Results
Performance metrics were obtained and illustrated for internal (Figure 1) and external validation (Table 1). In internal validation, the model discriminated ATTR-CM with an overall AUC of 0.871 (CI 95%: 0.868–0.874), consistent across patient subgroups. Notably, AUC was 0.901 (CI 95%: 0.895–0.908) in patients with Carpal Tunnel Syndrome. Next, external validation was addressed preliminarily since study recruitment remains active. Despite cohort variability, the overall performance was maintained, with an AUC of 0.861 (CI 95%: 0.859–0.863). At the optimal Youden's J threshold, Sensitivity was 74.52% (CI 95%: 73.55–75.48%), and Specificity 82.68% (CI 95%: 81.78–83.59%). Upon model optimization, external validation performance improved consistently as shown in Table 1.
Conclusions
These interim results suggest robustness and good generalizability from the AI model to identify ATTR-CM in ECGs in diverse, previously unseen populations. Recruitment completion will allow for subgroup analyses and further model optimization. Future research includes prospective use in relevant clinical settings.