Automatic echocardiographic image quality assessment and its impact in artificial intelligence and human measurements

Introduction

Image quality is crucial for echocardiography assessment, yet grading is subjective. With the growing adoption of artificial intelligence (AI), understanding how image quality impacts AI models has become essential.

Purpose

To develop a deep learning model, EchoQuality, to assess 2D echocardiography video quality.

Methods

EchoQuality was developed through supervised fine-tuning of the foundational model PanEcho (PAN) [1] using 6,475 videos annotated by three cardiologists. During a co-creation process, video quality was defined according to the definition of the boundary between blood pool and cardiovascular structures. A total of 5,057 videos were used for development and 1,418 for testing.

The impact of predicted quality on AI model performance was assessed for (i) disease classification and quantitative measurement (PAN and EchoPrime (EP) [2]) using 287,044 videos from 14,301 studies; (ii) view identification using Zhang (ZH) [3] and EP model in 6,697 videos from 464 studies; (iii) left ventricular (LV) segmentation using two models [4] trained on CAMUS [5] or EchoNet-Dynamic [6], evaluated on 1,969 frames from 100 CAMUS videos. Segmentation performance was assessed in terms of Dice score (DS). Additionally, the association between image quality and human reproducibility was evaluated for AHA region-level LV regional longitudinal strain (RS) and global longitudinal strain (GLS) in 587 videos from 100 studies.

Results

The inter-observer agreement for quality annotation was 56%, and EchoQuality achieved a test accuracy of 58%. Across 24 disease classifications and measurements, good-quality studies achieved the highest AUC and R² values in 18 (75%) metrics, whereas bad-quality studies showed the lowest performance in 20 (83%). For disease detections, the difference in AUC between good- and bad-quality studies was 0.05 (7% relative error reduction) for PAN and 0.03 (4%) for EP, while, for quantitative measurements, the difference in R² was 0.10 (26%) in PAN and 0.08 (10%) for LV ejection fraction prediction using EP. In view identification, good-quality videos were classified more accurately than bad-quality videos, with an improvement of 14.5% (18%) in the ZH model and 4.6% (5%) in EP. For LV segmentation, compared to low-quality ones, frames derived from good-quality videos showed a difference in DS of 0.05 (7.2%) for the EchoNet-trained model and 0.01 (1.7%) for the CAMUS–trained model (Fig 1).

Inter-video variability in RS and GLS was significantly associated with image quality, with good-quality images demonstrating lower variability compared with poor-quality (p < 0.001). Additionally, good-quality videos showed higher absolute RS and GLS values compared with poor-quality videos (p < 0.001) (Fig 2).

Conclusion

EchoQuality enables automated 2D echocardiography quality assessment, with strong potential for quality-aware selection of echocardiography videos for their use by AI models and humans.Figure 1Figure 2