Novice users of handheld cardiac ultrasound: comparative performance of two artificial intelligence guidance systems
M M Killalea, M A Gulati, J Kovac, C Kane, G Tsaban, J G Malins, D M Anisuzzaman, J I Jackson, J A Naser, T J Poterucha, P A Friedman, Z I Attia, J J Thaden, S V Pislaru, J G BirdAbstract
Background/introduction
Real-time artificial intelligence (AI) guidance may enable imaging-naïve users to acquire images suitable for diagnosis using handheld focused cardiac ultrasound (FoCUS).
Purpose
To compare image quality and diagnostic performance of novice-acquired FoCUS scans using two commercially available AI-guidance handheld FoCUS platforms.
Methods
In this prospective, randomised, crossover study, 128 adults referred for transthoracic echocardiography underwent back-to-back AI-guided FoCUS performed by one of three clinically and imaging-naïve (no imaging or clinical experience) operators after brief standardised training. Scan order of FoCUS System A (Vscan Air with AI guidance by Caption Health software) and B (Lumify with AI guidance by UltraSight software) was randomised at enrolment. Blinded experts graded image quality and interpreted findings. This included left ventricular ejection fraction (LVEF); an AI-LVEF algorithm also generated LVEF estimates. Diagnostic performance used an intention-to-diagnose approach (non-feasible examinations counted as incorrect) at LVEF thresholds <40% and <50% and for selected structural diagnoses. FoCUS acquisition and AI outputs were obtained blinded to clinical data and comparator tests. The reference standard: clinical transthoracic echocardiography.
Results
Participants were 59% male, aged 20-90 years, with body mass index 18-66 kg/m². Differences are shown in Figure 1. Users rated FoCUS System B higher for guidance usability and similarly for device usability (on a 5-point Likert). Images were graded as non-feasible (poor or missing) significantly more often for FoCUS System A versus with FoCUS System B for the parasternal long axis (17 vs 7%), apical 4 chamber (26 vs 8%), 3 chamber (46% vs 24%) and 2 chamber (46% vs 22%) views all p<0.01. Scan times were shorter for FoCUS System B 3.1±1.6 vs 3.6±1.6 mins for FoCUS System A, p<0.01, and quality scores better (3.5±0.7 vs 3.2±0.7, p=0.0001). Accuracy to detect disease was greater on images acquired on FoCUS System B than on FoCUS System A (Figure 2) – all p<0.001. Diagnostic accuracy for LVEF<50% was higher for System B than System A by both the AI algorithm (sensitivity: System B 90.9% vs System A 50.0%; specificity: System B 91.5% vs System A 84.0%) and expert interpretation (sensitivity: System B 95.5% vs System A 72.7%; specificity: System B 100% vs System A 96.2%).
Conclusion
Imaging-naïve novices can successfully acquire FoCUS images adequate for LVEF assessment using real-time AI guidance in two different guidance systems. However, performance differed substantially between systems, with System B (Lumify/UltraSight) delivering higher feasibility, image quality and intention-to-diagnose diagnostic performance, suggesting that the choice of handheld ultrasound device and AI-guidance solutions remains important when considering FoCUS screening pathways.Figure 2Figure 1