Patient-facing large language model assistants for chronic constipation education: A comparative study of expert evaluation, readability, and patient acceptability
Yanfen Gu, Shiqi Qiang, Zehui Xu, Haitao Zhang, Xiaoxiao Peng, Haiping Yu, Meidong XuObjective
To compare four patient-facing AI assistants for chronic constipation education across expert-rated quality, objective readability, and patient-reported acceptability.
Methods
In this two-phase study, 15 common questions and two clinical scenarios were submitted once to ChatGPT-4o, DeepSeek V3, Qwen3, and MedGo V1 using the same standardized Chinese prompt. Six blinded experts rated 68 responses across six domains. Objective readability was assessed using Chinese character count, reading difficulty score, and recommended reading age. In Phase 2, 30 patients each posed three questions; the investigator entered each question into all four assistants in a fixed sequence, and patients rated eight acceptability domains. Friedman tests were used; significant expert and patient results underwent Holm-adjusted Wilcoxon signed-rank tests.
Results
Accuracy (
Conclusion
Expert-rated quality, text complexity, and short-term patient acceptability were not fully aligned. No single assistant consistently outperformed the others. These findings describe model-specific performance profiles but do not establish clinical effectiveness, implementation readiness, or superiority of general-purpose versus medical-specialized systems.