Engagement Patterns with an AI Health Coach for Systemic Sclerosis Self‐Management: A Mixed Methods Study
Nirali Shah, Melanie Morris, Cristina Daraban, Sophia Johnson, Susan L. MurphyObjective
To evaluate utility of an artificial intelligence (AI) health coach for systemic sclerosis (SSc) self‐management and identify patterns associated with participant engagement.
Methods
We conducted a mixed‐methods study in which an AI health coach, powered by a large language model (LLM), was used to support self‐management for SSc. Twenty individuals with SSc interacted with the AI health coach over 4‐weeks. Quantitative usage metrics (number of conversations, user messages, and chat duration) were used to classify participants into high‐ and low engagement groups. Qualitative inductive content analysis of chat transcripts was used to identify the purpose of interactions. Quantitative and qualitative data were integrated using a joint display approach.
Results
Twenty participants (90% female; mean age 55 years) used the AI health coach for goals and strategies, information seeking, companionship, and disease monitoring. Participants with high engagement (N=8) had more coded interactions related to goals and strategies (23.6 vs 5.1), information seeking (10.8 vs 3.9), and companionship (7.9 vs 1.0) compared to those with low engagement (N=12). More participants in high engagement group used the AI health coach for companionship compared to the low engagement group (100% vs 33%). Exploratory analyses suggested greater improvements in fatigue (mean change −5.04 [95% CI −9.34, −0.73]) among high engagement participants, with no statistically significant between‐group differences.
Conclusion
Engagement with an AI health coach may not be fully captured by quantitative usage metrics alone. Engagement in our study was characterized by more frequent, action‐oriented, and companionship‐oriented interactions.