DOI: 10.32322/jhsm.1969053 ISSN: 2636-8579

Co-occurrence and clustering of bibliographic errors in large language model–generated endodontic references

Mehmet Adıgüzel
Show PDF Cite
Aims: Large language models (LLMs) are increasingly used in scientific writing; however, concerns remain regarding the reliability of the bibliographic references they generate. While previous studies have primarily focused on overall accuracy, the structural characteristics and distribution of errors remain insufficiently understood. This study aimed to evaluate the patterns, co-occurrence, clustering, and predictors of bibliographic inaccuracies in LLM-generated references in endodontics.Methods: A comparative experimental study was conducted using 2 LLMs, ChatGPT and Claude. Ten predefined endodontic queries were combined with 3 prompt types, and each model generated 5 references per query-prompt combination, resulting in 300 references. All references were verified using PubMed, Google Scholar, and CrossRef. Errors were categorized as digital object identifier (DOI) inaccuracies, title errors, journal mismatches, author inconsistencies, and year errors. Error burden, co-occurrence patterns, DOI error clustering, and predictors of DOI error occurrence were analyzed using Chi-square tests, multivariable logistic regression, and a generalized linear mixed-effects model.Results: DOI errors were the most frequent inaccuracy, followed by title and journal errors. ChatGPT generated a higher proportion of multiple-error references than Claude (48.7% vs 6.0%, p

More from our Archive