Performance of an artificial intelligence-based smartphone application to assess frailty in older patients with heart failure
S Tahara, T Nagai, T Tamura, M Nakao, Y Wakamatsu, Y Kato, J Matsumoto, H Hosoda, H Nakano, Y Watanabe, Y M Ito, K Hirata, T Ogawa, T AnzaiAbstract
Background
Although frailty is strongly associated with adverse outcomes in patients with heart failure (HF), most conventional frailty assessment tools contain substantial subjective components and are therefore prone to considerable interrater variability. We previously developed an artificial intelligence (AI) model that estimates the Clinical Frailty Scale (CFS) from gait videos with high agreement versus expert consensus. Whether this model maintains performance when implemented as a smartphone application enabling real-world point-of-care assessment has not been prospectively tested.
Purpose
To evaluate agreement between an AI-based smartphone frailty assessment system and expert-adjudicated CFS in older patients with HF.
Methods
This prospective multicenter investigator-initiated validation trial was conducted across six hospitals between January and March 2025. Patients aged 75 years or older with symptomatic chronic HF were enrolled. Gait videos were recorded using a smartphone camera on iPhone 15 Pro while patients walked along an L-shaped 4 m course (Figure 1A). The prespecified model was fixed before trial initiation without retraining. The reference standard was the centrally adjudicated CFS determined by eleven trained cardiologists using a modified Delphi process based on gait videos and standardized activities of daily living data. The primary endpoint was agreement between AI-derived and adjudicated CFS assessed by quadratic weighted kappa (QWK). Superiority was defined as the lower bound of the two-sided 95% confidence interval (CI) exceeding the predefined threshold of 0.72, reflecting expert interrater agreement from our prior study. Assuming an expected QWK of 0.85 with one-sided alpha 0.025 and power 0.80, the target sample size was 430. Secondary endpoints included test reproducibility and subgroup agreement by age, sex, and HF severity.
Results
A total of 462 patients were enrolled. The application demonstrated a high technical success rate of 99.3% in this real-world setting. The full analysis set and per-protocol set included 442 and 429 patients. Mean age was 81.8 ± 5.0 years, 55.4% were male, 19.7% had ischemic heart disease, and 12.4% had NYHA class III. Agreement between AI-derived and adjudicated CFS exceeded the superiority threshold, with QWK 0.78 (95% CI 0.74 to 0.83) in the full analysis set and QWK 0.78 (95% CI 0.74 to 0.82) in the per-protocol set (Figure 1B and C). Discrepancy of two or more CFS grades occurred in 0.5%. In reproducibility analysis using two consecutive measurements, agreement was high with QWK 0.89 (95% CI 0.86 to 0.92). Performance was consistent across age groups, sex strata, and across HF severity strata (Figure 2A-C).
Conclusion
The AI-based smartphone application provided rapid frailty grading with substantial agreement compared with expert-adjudicated CFS in older patients with HF, supporting its use for standardized point-of-care objective frailty assessment.