Comparison of Gram stain interpretation accuracy between a computer-aided diagnosis app and microbiology specialists in blood culture samples

Gram staining is one of the basic tests to identify the organism in microbiological laboratory, especially in blood culture. However, microbiological specialists are not always available, which can lead to treatment delays. Therefore, we developed a computer-aided diagnosis (CAD) system that uses artificial intelligence to predict causative pathogens from Gram-stained blood culture images, focusing on both morphological (Class 1) and detailed (Class 2) classification. In this retrospective observational study, we compared the accuracy of predictions made by microbiology specialists (MS) and by CAD using iPhone-captured images of clinical samples stained with the Bartholomew and Mittwer method collected from two tertiary care hospitals between 1 October 2022 and 31 January 2023. Among 126 samples (378 images) for aerobic bottles (AE) and 90 samples (270 images) for anaerobic bottles (AN), the accuracy of MS and CAD prediction (95% confidence interval) was 91.5% (90.5%–92.3%) and 75.4% (70.7%–79.7%) for AE and 90.1% (88.9%–91.2%) and 76.7% (71.2%–81.6%) for AN, respectively. Although we could not demonstrate non-inferiority, most of the mispredictions occurred with Gram-negative cocci, Haemophilus influenzae , and Campylobacter spp., while relatively good results were obtained for other species. CAD has the potential to provide early diagnostic results and indicate timely treatment during off-hours and in medical facilities without microbiology specialists. On the other hand, it should be noted that certain bacterial species remain challenging for AI-based identification, and improving accuracy in these areas is an important task for future development.

IMPORTANCE

Gram staining of a positive blood culture is essential for early diagnosis and appropriate antibiotic selection, but interpretation requires trained specialists, limiting rapid reporting. While artificial intelligence (AI)-based automation has been explored, most previous studies either focused on a limited range of bacterial species or evaluated only the performance of the device itself. Also, most systems rely on bulky equipment. To address this, we developed a mobile device-based computer-aided diagnosis system for Gram stain interpretation and conducted a non-inferiority trial comparing its accuracy with that of microbiology specialists. Although non-inferiority was not demonstrated, we identified bacterial species that the AI system had difficulty distinguishing. With an appropriate understanding of these limitations, AI-assisted Gram stain interpretation could help reduce turnaround time and support timely clinical decision-making.