The accuracy, completeness, and reliability of ChatGPT responses on cesarean section care for new parents during puerperium
Ya-Ting Kuan, Chang-Ching Yeh, Chiu-Ping Chuang, Shu-Fen Chen, Hsiu-Ju Chang, Chi-Wen Chen, I-Ching HouBackground
Rising cesarean section (C-section) rates and shortened hospital stays pose challenges to traditional postpartum care models. The growing use of natural language processing models (e.g., ChatGPT) for medical information, coupled with variability in response quality, poses risks when medical information is not verified. Inaccurate information or unproven remedies may mislead new parents, potentially resulting in delayed medical care or even life-threatening consequences.
Objectives
We aimed to evaluate the accuracy, completeness, and reliability of ChatGPT responses to postpartum self-care questions raised by new parents following a C-section, as assessed by healthcare professionals.
Methods
This multiphase observational study comprised qualitative dataset development followed by a cross-sectional expert evaluation. We developed 75 questions from interviews with new parents who sought C-section–related postpartum self-care information, and 10 obstetric healthcare professionals evaluated ChatGPT-generated responses for accuracy using a 6-point Likert scale and completeness using a 3-point Likert scale. We analyzed the data using the Kruskal–Wallis test with Dunn–Bonferroni post hoc comparisons.
Results
The median accuracy score was 5 (range: 1–6), indicating generally high accuracy (mean = 5.15, 85.83%). The median completeness score was 2 (range: 1–3), indicating generally adequate completeness (mean = 2.55, 85%). Accuracy and completeness scores were moderately and positively associated (p < .001).
Conclusion
ChatGPT generated largely accurate information in response to diverse medical queries, as judged by obstetric healthcare professionals, although limitations were observed. These findings highlight its potential as a supplementary educational resource rather than a substitute for individualized professional medical advice.