Analyzing Online Discourse of Retrograde Cricopharyngeal Dysfunction: A Human‐Validated LLM Study
Spencer T. Johnson, Nancy JiangAbstract
Objectives
Perform a human‐validated large language model (LLM) analysis of an online Reddit community dedicated to retrograde cricopharyngeal dysfunction (R‐CPD), characterizing common themes, symptoms, treatment experiences, and diagnostic barriers.
Study Design
Cross‐sectional, observational analysis of patient‐generated online discourse using LLM‐based thematic classification validated against human adjudication.
Setting
r/NoBurp, a public Reddit community dedicated to R‐CPD.
Methods
All posts from March 2014 to December 2025 were extracted and deduplicated. A 400‐post subset was human‐adjudicated to establish a reference standard for interrater comparison (precision, recall, F1 score, Cohen's kappa). All posts were analyzed with ChatGPT‐5.2 for theme identification, temporal trends, symptoms frequency, and sentiment analysis. A bibliometric analysis compared annual R‐CPD publication volume to annual Reddit post volume.
Results
20,299 unique posts were analyzed. The five dominant themes were core symptom burden (56.5%), treatment experience (53.3%), healthcare navigation (26.0%), self‐management (15.2%), and reflux‐related symptoms (12.5%). Annual post volume rose sharply years before the major increase in R‐CPD‐related publications. Interrater agreement was high for botulinum toxin/procedural treatment ( k = 0.70, 95% CI [0.64‐0.77], F1 = 0.81), moderate for core symptom burden ( k = 0.44, 95% CI [0.36‐0.52], F1 = 0.77), and slight‐to‐fair for diagnostic journey/healthcare navigation ( k = 0.22, 95% CI [0.15‐0.29], F1 = 0.38).
Conclusion
Patients experience prolonged diagnostic journeys marked by clinician unfamiliarity with R‐CPD, often identifying their condition through online communities. Rising patient discourse preceded growth in the published literature, suggesting that r/NoBurp functioned not only as a support forum but also as an accelerator of R‐CPD research.