Enhancing Mental Health Support Using Fine-Tuned Large Language Models: A Comparative Evaluation of LLaMA 2 and LLaMA 3

The growing prevalence of mental health disorders, combined with limited access to traditional therapeutic services, has accelerated the demand for scalable and empathetic AI-based interventions. This study explores the development and evaluation of an AI-powered chatbot designed to provide mental health support using advanced Large Language Models (LLMs), specifically LLaMA 2 and LLaMA 3. Using a mental health counseling dataset, the models were fine-tuned on counseling context–response pairs and evaluated for semantic and lexical similarity to reference responses. The training pipeline incorporated quantization, LoRA adaptation and a structured prompt template. The models were evaluated using BERTScore, BLEU, ROUGE, and METEOR, offering a multi-dimensional view of linguistic and semantic quality. Results show that LLaMA 3 achieved a higher mean BERTScore F1 than LLaMA 2 (0.860 versus 0.835). The results characterize the fine-tuned models’ performance under automatic text-similarity evaluation. Human evaluation is required before conclusions can be drawn about empathy, safety or suitability for mental health support. Ethical considerations and future directions, including user validation and multimodal expansion, are also discussed.