Deep learning analysis of contactless facial video and remote photoplethysmography for acute heart failure detection
Y Teo, M Zhou, B Wu, L Zhen, X Wang, T H Tng, L Yong, S M Goh, X Ong, M Chan, C H Sia, T LiAbstract
Background and Purpose
Early identification of acute decompensated heart failure (HF) is essential to enable timely treatment and prevent clinical deterioration. Current diagnostic tools require blood testing or imaging, limiting rapid, scalable screening. We evaluated the feasibility of contactless camera-based photoplethysmography (rPPG) derived from facial video for detection of decompensated HF.
Methods
In this prospective cohort study, 275 hospitalized patients from July 2024 to February 2026 were enrolled, including 87 with acute decompensated HF (irrespective of ejection fraction) and 188 controls admitted for non-HF conditions. A standardized 2-minute facial video was recorded during index admission and at 1-month outpatient clinic follow-up. Videos were segmented into 10-second clips and processed using facial region detection, motion stabilization, and remote PPG signal extraction, which have been previously validated for non-contact cardiovascular signal measurement [1–3]. To prevent information leakage, dataset partitioning was performed at the participant level, in accordance with best practices for clinical machine learning studies. After stratification, patients were randomly allocated into training (80%) and validation (20%) sets. HF classification was performed using a PhysNet-based deep neural network. Final model performance was evaluated on the validation set.
Results
Baseline characteristics are summarized in Table 1. The overall cohort had a median age of 64 years and was predominantly male (74.9%). There were no significant differences between HF and control groups with respect to age, sex distribution, body mass index, diabetes mellitus, dyslipidaemia, hypertension, or prior acute myocardial infarction (all P>0.05).Patients with HF were more frequently current smokers (32.2% vs 14.4%, P=0.002) and less frequently non-smokers. Among HF patients, mean left ventricular ejection fraction was 47%, reflecting inclusion across the ejection fraction spectrum. Both video-based and rPPG-based approaches demonstrated comparable discriminative performance. The video-based model achieved an accuracy of 0.82, AUPRC 0.78, F1-score 0.80, and AUROC 0.76. The rPPG-based model achieved an accuracy of 0.78, AUPRC 0.80, F1-score 0.82, and AUROC 0.74. These findings support the feasibility of extracting diagnostically relevant cardiovascular signatures from facial video for HF detection.
Conclusion
Contactless facial video analysis using rPPG enables non-invasive identification of decompensated HF. This approach may support scalable screening and remote monitoring applications.Table 1 baseline characteristics table