AI vs. Humans for Online Support: Comparing the Language of Responses from LLMs and Online Communities of Alzheimer’s Disease

AI chatbots are increasingly integrated into various sectors, including healthcare. We examine their role in responding to queries related to Alzheimer’s Disease and Related Dementias (AD/ADRD). We obtained real-world queries from AD/ADRD online communities (OC)—Reddit (r/Alzheimers) and ALZConnected. First, we conducted a small-scale qualitative examination where we prompted ChatGPT, Bard, and Llama-2 with 101 OC posts to generate responses and compared them with OC responses through inductive coding and thematic analysis. We found that although AI can provide emotional and informational support like OCs, they do not engage in deeper conversations, provide references, and share personal experiences. These insights motivated us to conduct a large-scale quantitative examination of comparing AI (GPT) and OC responses (90K) to 13.5K posts, in terms of psycholinguistics, lexico-semantics, and content. AI responses tend to be more verbose, readable, and complex. AI responses exhibited greater empathy, but more formal and analytical language, lacking personal narratives and linguistic diversity. We found that various LLMs, including GPT, Llama, and Mistral, exhibit consistent patterns in responding to AD/ADRD-related queries, underscoring the robustness of our insights across LLMs. Our study sheds light on the potential of AI in digital health and underscores design considerations of AI to complement human interactions.