DOI: 10.37349/edht.2026.1011105 ISSN: 2996-9409

Hierarchy-aware strategy recommendation for emotional support dialogues via hyperbolic representation and conservative offline learning

Bailing Zhang, Ruofan Zhang
Show PDF Cite
Aim: Effective emotional support requires selecting support strategies appropriate to the seeker’s state and grounded in counselling theory. We present a computational proof-of-concept that combines hyperbolic representation learning with conservative offline reinforcement learning for support-strategy recommendation. Methods: Support strategies from Hill’s Helping Skills framework are embedded in the Poincaré ball to encode their three-level hierarchy. On this representation, we train Conservative Hyperbolic Implicit Q-Learning (CH-IQL) on the Emotional Support Conversation dataset (ESConv; 1,300 dialogues; 910/195/195 train/validation/test split). All results are reported as mean ± standard deviation over five training seeds. Policy value is measured by independent Fitted Q-Evaluation (FQE) with twin Q-networks, calibrated against the behavior policy’s Monte-Carlo ground truth and an analytic value ceiling, with dialogue-level bootstrap confidence intervals. Strategy appropriateness is additionally assessed by four blinded human raters over 120 stratified test states. Results: Under the stochastic (sampling) deployment, CH-IQL attains an FQE-estimated value of 4.79 ± 0.07 versus 3.66 for the estimated behavior policy—positive under every seed and under two alternative, feedback-anchored reward designs—while its strategy distribution matches the entropy of human supporters (1.98 vs. 1.99) and uses all eight strategies. In blinded pairwise comparison, raters preferred the model’s strategy in 43.9% of disputed cases (95% CI 32.6–55.9%), statistically indistinguishable from the observed supporter behavior. The heuristic reward shows no dialogue-level correlation with seeker-reported outcomes (Spearman ρ = 0.009), which we disclose and address through reward-robustness experiments. Conclusions: The framework improves the specified reward while remaining comparable to observed supporter behavior under expert judgment. These results support offline reinforcement learning with hierarchical representations as a viable strategy-recommendation mechanism, and identify outcome-correlated reward design as the key requirement for progress toward real-world utility. The system is a computational proof-of-concept and is not suitable for clinical deployment.

More from our Archive