LLM-Assisted Reinforcement Learning for Affective Game Adaptation EICS029

Adaptive games increasingly combine player modeling, affect sensing, and runtime control, but evidence for how large language models (LLMs) and Reinforcement Learning (RL) could participate in such loops remains limited. We present an affect-aware game adaptation architecture that combines a warm-started tabular Q-learning controller with bounded LLM-assisted action suggestion inside a MAPE-K loop. The LLM is not allowed to act freely: it can only recommend one action from a fixed, predefined adaptation menu, while reinforcement learning remains responsible for value updates and policy improvement. We implemented the architecture in CubeWars, a top-down shooter instrumented with browser-side facial-expression sensing and four concrete adaptation levers: difficulty, zombie color, background audio, and modal gameplay prompts. We evaluated three within-subject conditions with 46 participants: a static baseline, RL-only adaptation, and LLM-assisted RL adaptation. The strongest objective result is that both adaptive conditions improved absolute progression outcomes relative to the static baseline, whereas RL-only and LLM-assisted RL did not differ significantly on normalized objective throughput measures. The clearest hybrid-specific benefits were experiential: the LLM-assisted condition produced higher engagement, stronger absorption, and greater awareness of adaptation than the RL-only condition. For affect, the hybrid condition showed the highest mean value on an exploratory reward-relevant emotion composite, but the pairwise hybrid-versus-RL-only contrast did not survive Bonferroni correction; we therefore interpret the affective findings primarily through their per-emotion decomposition and event-level analyses. The paper contributes: 1) a pattern for integrating constrained LLM advice into a self-adaptive interactive system without replacing the underlying RL controller; 2) an artifact-grounded implementation of affect-driven runtime game adaptation in a playable game; and 3) an empirical evaluation showing that, in the present evidence, the main added value of the hybrid design lies in perceived responsiveness and player experience.