DOI: 10.4103/jehp.jehp_1203_25 ISSN: 2277-9531

Evaluating ChatGPT-4 omni’s performance on the Iranian dental board examination across specialties: A cross-sectional study

Hossein Esmailpour, Mehran Falahchai, Yasamin Babaee Hemmati, Yazdan Haghi
Show PDF Cite

BACKGROUND:

The role of Artificial Intelligence (AI) chatbots in dental education is growing, yet their reliability in specialty-level knowledge remains uncertain. This study was conducted to evaluate ChatGPT-4 Omni’s performance using Iranian National Board Dental Examination questions (NBDE).

MATERIALS AND METHODS:

In this cross-sectional study conducted in Iran in 2024-2025, 100 multiple-Choice questions from the Iranian NBDE across 10 specialties were included ( n = 1000). Each question was translated into English using ChatGPT and was reviewed by two bilingual dental specialists. Each question was submitted to ChatGPT-4 Omni. A passing threshold of 70% accuracy rate was considered for each specialty exam. Statistical analysis was performed using the Chi-square test in MedCalc (version 23.0.6; MedCalc Software Ltd) (α = 0.05).

RESULTS:

The performance of ChatGPT-4 Omni varied across specialties. The highest accuracy rate was seen in pathology, where 73 out of 100 questions were answered correctly in both Farsi and English versions. The lowest performance was observed in orthodontics (Farsi: 31%, English: 40%) and prosthodontics (Farsi: 37%, English: 32%). In multilingual assessment, no statistically significant differences were found between Farsi and English responses in any specialty (e.g, P = 0.671 for Oral Surgery, P = 0.463 for Oral Medicine).

CONCLUSIONS:

ChatGPT-4 Omni demonstrated limited proficiency in answering Iranian NBDE questions. Further domain-specific fine-tuning may be necessary before integrating it into clinical decision-making or dental education.

More from our Archive