Comparing AI-ChatGPT and clinician-authored cancer patient education materials: A cross-sectional assessment
Runcie C.W. Chidebe, Krishna Prasad Sapkota, Sampson C. Ipiankama, Deborah N. Ejemole, Leigh Leibel, Faridah M. Banwo, Fabio Ynoe de MoraesAbstract
Objectives:
In Nigeria and Africa, the use of artificial intelligence (AI) large language models such as ChatGPT as a source of health and cancer information is increasing; however, the quality of this information remains largely unknown. This study assessed whether ChatGPT-generated cancer patient education material (PEM) aligns with clinician-authored content in usefulness, understandability, and clarity.
Methods:
Using a cross-sectional study design, 409 adults were recruited to complete questionnaires assessing the usefulness, understandability, and clarity of cancer PEMs created by AI-ChatGPT (using GPT-4o) and clinicians.
Results:
Of 371 participants who completed the questionnaire, most were female (70%), held a bachelor’s degree or higher (90%), were urban residents (80%), and over half (58%) lacked health insurance. The findings showed that ChatGPT-using GPT-4o–produced PEMs were comparable to clinician-developed materials in usefulness, understandability, and clarity, with no overall significant differences. However, clinician statements on “healthy diet” and “limit alcohol” were rated clearer than AI’s equivalents, with statistically significant differences for both ( p = 0.025, d = 0.24; p = 0.006, d = 0.30). Only 14% of participants reported using AI for health information, yet 58% expressed trust in it, suggesting potential for rapid adoption.
Conclusions:
ChatGPT (GPT-4o) can produce PEMs comparable to clinician-created ones in usefulness, understandability, and clarity. This is the first Africa-based study to compare clinician- and AI-generated cancer PEMs.
Practice implications:
Clinicians and health educators may consider using AI, particularly ChatGPT, to support PEM development in Africa; however, outputs require clinician review, cultural adaptation, and alignment with the health system.