DOI: 10.38053/acmj.2002163 ISSN: 2718-0115

Can large language models adhere to clinical guidelines in tooth avulsion management? A repeated-measures comparative study

Banu Çiçek Tez Yaşar, Meltem Karahan, Arif Yiğit Güler, Bahar Başak Kızıltan Eliaçık
Show PDF Cite
Aims: To comparatively evaluate adherence to the 2020 International Association of Dental Traumatology (IADT) guidelines and short-term temporal consistency across three large language models (LLMs)-ChatGPT (GPT-5.5), Gemini 2.5 Flash, and Microsoft Copilot-when answering open-ended questions on tooth avulsion management.Methods: Fourteen open-ended questions were categorized into four domains: Emergency and first aid, clinical management, prognosis and critical factors, and advanced decision-making. Each question was submitted to each LLM three times daily for three consecutive days. Two pediatric dentists classified responses as correct or incorrect by consensus using a guideline-based answer key. Cochran’s Q and post hoc McNemar tests were used for comparisons.Results: Among 378 responses, ChatGPT generated the highest number of correct responses (94/126; 74.6%), followed by Gemini (87/126; 69.0%) and Copilot (79/126; 62.7%). The overall intermodel difference was significant (p=0.021), with a significant pairwise difference only between ChatGPT and Copilot (p=0.015). No significant differences were observed in emergency and first aid, clinical management, or prognosis and critical factors (p>0.05). Advanced decision-making differed significantly among models (p=0.008), particularly between ChatGPT and Gemini (p=0.016). No significant within-day or across-day variation was detected (p>0.05).Conclusion: The evaluated LLMs showed no statistically significant short-term temporal variation, but their adherence to IADT recommendations varied across models and clinical domains. ChatGPT achieved the highest performance, while the greatest divergence occurred in the predefined Advanced Decision-Making domain, specifically in questions concerning extra-alveolar dry time and delayed replantation. These models should be used only as adjunctive tools, with recommendations verified against guidelines under professional supervision.

More from our Archive