A unified framework for sentiment classification and topic modeling of mHealth app reviews
Naashrah Naashidin Preetasha, Nabeel Mohammed, Silvia Ahmed, M M Lutfe Elahi, Sifat MomenObjective
Mobile health (mHealth) app reviews contain large-scale evidence about user satisfaction, usability issues and unmet needs. However, extracting actionable insights remains challenging because app reviews are short, noisy, domain-specific and often sentiment-imbalanced. This study proposes a unified framework for sentiment classification and topic modeling of mHealth app reviews.
Methods
User reviews were collected from six internal health-related applications across Google Play and the Apple App Store, with one additional external application collected for external validation. After preprocessing, sentiment labels were derived from app ratings and the task was reformulated as binary positive-versus-negative classification. We evaluated classical machine learning classifiers with Word2Vec and contextual sentence embeddings, BERT-based transformers, generative large language models (LLMs), and the tabular foundation model TabPFN. BERTopic with LLM-assisted labeling was used to identify recurring topics, and LIME and attention heatmaps were used to inspect model behavior through illustrative examples.
Results
Contextual sentence embeddings improved classical machine learning performance, with Support Vector Machine reaching 93.32% accuracy and 91.49% macro-F1. Fine-tuned LLaMA-3.2 achieved the strongest internal performance with 96.09% accuracy and 95.00% macro-F1, while TabPFN achieved 92.34% accuracy without task-specific parameter training. In subset-specific external validation, fine-tuned LLaMA-3.2 achieved the highest observed performance across the platform-transfer subset and the application-transfer subsets. Topic modeling identified concerns related to usability, billing and cancellation, account access, medicine delivery and personalized health management.
Conclusion
The proposed framework combines high-performing sentiment prediction, topic-based insights, qualitative inspection of model behavior and proof-of-concept deployment to support scalable user-centered evaluation of mHealth applications.