Early Prediction of Silent Cardiac Death Using Cross Modal Attention Based Multimodal Diagnostic Framework for Clinical Evaluation

ABSTRACT

Silent cardiac death is a serious health problem after the COVID‐19 pandemic, as it happens when the heart unexpectedly stops beating. This type of sudden death is commonly triggered by the severity of cardiac‐affecting factors such as blood oxygen saturation level, blood pressure, SpO ₂ , BMI, cholesterol levels, and diabetes. This paper presents a cross‐modal attention cardiac multimodal data fusion approach for the prior prediction of silent cardiac death using the fusion of 12‐lead electrocardiography image features and electronic health records. The study includes 380 cardiac samples for 14 EHR parameters obtained along with 12‐lead ECG images for a multimodal database, collected from the CardioHTDC. In proposed models, a 2D ResNet‐50 architecture with 16 residual bottleneck blocks is used to extract deep spatial features from 12‐lead ECG images using 2D convolutional operations. The BERT framework for cardiac EHR data uses the sequential masked token prediction technique. These cardiac multimodal data features are concatenated into the cross‐modal attention‐based technique. These multimodal architectures are also tested on unseen data generated using the ADS1298ECG‐FE kit embedded in the CardioSim II ECG simulator with data acquisition performed through the ADS1298ECG‐FE evaluation software and compared with the VGG16 model and EfficientNet‐B0‐based multimodal architecture performance. The ResNet50 and BERT multimodal fusion approach is classified into five classes with the highest accuracy of 94.44%, with a precision of 94.57% and recall of 94.41% on the CardioHTDC dataset and achieved an accuracy of 91.20% on the CardioSim system‐generated dataset. The proposed cross‐attention multimodal diagnostic framework effectively integrates ECG imaging and EHR‐based clinical attributes to improve early SCD prediction. Future work will focus on expanding the dataset and optimizing multimodal fusion for broader clinical use.