Influence of artificial intelligence on the performance of endosonographers in detection of solid pancreatic tumors: a tandem randomized video study

Abstract

This study aimed to evaluate the influence of a pre-commercial artificial intelligence (AI)-based software system on endosonographers’ performance and diagnostic confidence in the detection of solid pancreatic tumors (SPTs).

A randomized controlled tandem video study was conducted and included 38 curated endoscopic ultrasound (EUS) videos from three experiments reflecting varying levels of perceptual difficulty. Each participant evaluated every video clip twice (with and without AI assistance), using a counterbalanced 1:1 randomized design. Assessment consisted of detecting SPTs on a paused frame at the end of each clip and localizing the suspected lesion on a grid-based sheet. Diagnostic confidence was recorded. All assessments were compared against an expert-defined reference standard; true-positive cases were histologically confirmed lesions. The primary end point was diagnostic accuracy. Secondary end points included stand-alone AI performance, the effect of AI by experience level, and diagnostic confidence.

Among 57 participating endosonographers, diagnostic accuracy improved from 70.8% without AI to 77.1% with AI (P < 0.001). Stand-alone AI accuracy was 92.3%. AI increased diagnostic accuracy by 6.3 percentage points, with the benefit mainly observed in high-complexity clips (small and transient lesions) and no significant improvement in lower-difficulty scenarios. No statistically significant difference in AI effect was detected between experience levels. AI assistance also increased diagnostic confidence, reflected by a significant shift toward higher confidence ratings (P < 0.001).

AI assistance improved diagnostic accuracy and confidence in detecting SPTs on EUS video sequences, particularly for small and briefly visible lesions, supporting its potential role as a complementary tool in pancreatic EUS.