Evaluation of artificial intelligence–generated patient education content for adolescent idiopathic scoliosis: A comparative study of reliability, educational value and readability
Yekta Furkan Altın, Murat Korkmaz, Bedirhan Demir, Tolga Yüksel, Turgut Akgül, Merve Damla KorkmazObjective
To compare the reliability, educational value, readability, and exploratory clinical accuracy/safety of patient-education responses generated by four consumer AI chatbots for adolescent idiopathic scoliosis (AIS).
Methods
Ten AIS questions were submitted once, in separate conversations, to ChatGPT (GPT-5, free version), Gemini (Gemini 2.5 Flash, free version), Claude (Claude Sonnet, free version), and DeepSeek (DeepSeek-V3, free version) between 27 May and 10 June 2026. Three blinded orthopedic spine surgeons evaluated first responses using mDISCERN, PEMAT-Understandability, PEMAT-Actionability, and GQS; readability was assessed with five standard indices. Because all four models answered the same 10 questions, model comparisons were analyzed as question-matched data. An exploratory assessment evaluated factual inaccuracies, clinically important omissions, potentially harmful recommendations, and SRS/SOSORT concordance.
Results
Forty responses were analyzed. Question-matched comparisons showed significant differences among models for mDISCERN, PEMAT-U, PEMAT-A, and GQS (all Friedman p<0.001), with ChatGPT achieving the highest mean scores (4.03±0.51, 84.07±1.90, 83.07±2.07, and 4.33±0.35, respectively). Responses generally required a 10th-to 11th-grade reading level. Factual inaccuracies occurred in 5/40 responses (12.5%), clinically important omissions in 2/40 (5.0%), and potentially harmful recommendations in 2/40 (5.0%); matched exploratory comparisons were not statistically significant. All responses were at least partially SRS/SOSORT-concordant by majority classification.
Conclusion
AI chatbots differed substantially in expert-rated AIS educational quality, while exploratory accuracy/safety differences were not statistically significant. Readability remained above recommended patient-education levels. AI-generated information should supplement, not replace, clinician-guided counseling.