Electrocardiogram Abnormality Classification From Summary Statistics With Patient‐Level Validation and Process‐Focused Evaluation
Sungjoon Hong, Christina Hartnett, Michael Ruane, Jason Tagliarino, Michael Nizich, M. TomaABSTRACT
Background
Machine learning (ML) applications in clinical medicine are vulnerable to data leakage, particularly temporal leakage from post‐diagnostic features and patient‐level leakage from improper partitioning, compromising electrocardiogram (ECG) abnormality detection systems. This study addresses these vulnerabilities through patient‐level data splitting and systematic evaluation across multiple classification scenarios.
Methods
ECG data from 4419 observations representing 2180 unique patients were analyzed using Random Under‐Sampling Boosting (RUSBoost). Three binary classification scenarios were defined: Scenario 1 (Abnormal vs. Normal, excluding borderline), Scenario 2 (Abnormal + Borderline vs. Normal), and Scenario 3 (Abnormal vs. Normal + Borderline). Patient‐level stratified partitioning (60:20:20) prevented information leakage. Models were evaluated using accuracy, sensitivity, specificity, F1‐score, learning curves, and precision‐recall curves across all partitions.
Results
Scenario 1 demonstrated optimal generalization with test accuracy of 70.79%, sensitivity of 60.30%, specificity of 86.50%, and F1‐score of 0.712, exhibiting monotonic decline across partitions (73.98%→71.50%→70.79%). Scenario 2 achieved 65.26% test accuracy with greater degradation (6.17 percentage points), reflecting increased difficulty when borderline cases were grouped with abnormal findings. Scenario 3 exhibited problematic non‐monotonic patterns (71.84%→67.71%→69.36%) with premature convergence, indicating fundamental generalization challenges despite near‐balanced classes (1.07:1 ratio).
Conclusions
Aggregate metrics alone inadequately support clinical deployment decisions. Medical AI evaluation must examine learning dynamics, generalization stability, and precision‐recall calibration across partitions. Process‐focused standards assessing monotonic decline, convergence characteristics, and calibration stability are essential for reliable ECG abnormality detection deployment.