Differential Diagnosis Performance of
ChatGPT
in Oral and Maxillofacial Pathology
Sohaib Shujaat, Marryam Riaz, Jawaher Mohammed Alsubaie, Aljwhrah Abdulaziz Almuammar, Arun Pillai, Abdulmohsen Alfadley, Ali Anwar Aboalela ABSTRACT
Objective
To evaluate ChatGPT (GPT‐5.1) for ranked differential diagnoses in oral and maxillofacial pathology, assess image effects on diagnostic performance and examine output stability across repeated runs.
Methods
This retrospective diagnostic accuracy study included 160 biopsy‐confirmed cases. Cases were evaluated using a standardised prompt under two conditions: text‐only and text plus clinical photographs and/or radiographs. Each case underwent three independent runs. Performance was measured using Top‐1, Top‐3, Top‐5 and Top‐10 accuracy.
Results
Top‐1 accuracy was 44.0% for text‐only and 43.3% for text + image inputs. Accuracy increased at broader thresholds, with Top‐10 accuracy reaching 80.7% and 82.9%, respectively, but no significant differences were observed between input conditions at any Top‐K threshold. Top‐1 performance was stable across runs, although variability was observed at broader Top‐K thresholds in the text‐only condition.
Conclusions
ChatGPT ranked the reference diagnosis first in fewer than half of cases, while broader differential lists more often included it. Images did not significantly improve accuracy, although heterogeneous and non‐standardised visual inputs may have limited detection of multimodal effects. These findings support further investigation for hypothesis generation and educational use rather than clinical decision support or replacement of expert judgement and histopathological confirmation.