P0407 Use of social listening to explore diet-related commentary from Inflammatory Bowel Disease social media groups

Background

People with inflammatory bowel disease (IBD) desire dietary information1 and use social media to fill the information gap.2 We aimed to capitalise on the large volume of commentary available in social media groups to determine frequently asked dietary questions and nutrition-related topics regarding this condition, utilising the Chat Generative Pre-trained Transformer 4 model (ChatGPT-4) to facilitate social listening analysis.

Methods

Following ethical guidelines and the approval of Facebook group administrators, social listening techniques were used to extract 11,945 online community comments (2021-2023). All data was deidentified and cleaned according to ethical standards, ensuring only context-specific comments were included for analysis. The data was analysed using content and thematic analysis and ChatGPT-4. Duplicate thematic analysis explored the meaning of the comments. The ChatGPT-4 large language model was trained on 10% of manually coded data using the few-shot learning technique.3 Results from both manual and ChatGPT-4 coding were manually reviewed for congruence to ensure the model’s analysis was suitable and could then be applied to all data.

Results

Moderate percentage agreement existed between manual and model coded data (75%). Medical therapy (31.2%) and diet-related comments (37.5%) dominated the conversations. A total of 350 unique diet-related queries emerged. Content analysis of these indicated the most commonly mentioned diet-related topics were management of gastrointestinal/extraintestinal symptoms (37.8%), natural therapies/dietary approaches to manage symptoms (20%) and maintenance of nutritional status (5.6%). Thematic analysis indicated five interrelated themes: desire for support and connection; exasperation and desire for information; open to considering advice from non-experts; desire for food to be thy medicine and IBD holds me back.

Conclusion

Using ChatGPT-4-enabled social listening of IBD Facebook groups, verified with traditional qualitative techniques, improved efficiency in handling large volumes of qualitative information and identified key dietary concerns prevalent in online discussions. Future studies should establish methodological and reporting standards for artificial intelligence-enabled social listening to ensure comparability between studies. With respect to individuals with IBD, Facebook groups provided immediate access to information and connection, compensating for the perceived lack of timely access to a knowledgeable and empathetic multidisciplinary care team. Improving access to specialized professionals, including dietitians well-versed in IBD, could reduce reliance on these platforms as a primary source of information.

References

1.Miglioretto C, Beck E, Lambert K. A scoping review of the dietary information needs of people with inflammatory bowel disease. Nutr Diet. 2024;81(1):79-93.

2.Ni Z, Zhu L, Li S, Zhang Y, Zhao R. Characteristics and associated factors of health information-seeking behaviour among patients with inflammatory bowel disease in the digital era: a scoping review. BMC Public Health. 2024;24(1):307 - 22.

3.Chew R, Bollenbacher J, Wenger M, Speer J, Kim A. LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding. arXiv. 2023:1-27.