Seed Variance and Evaluation Rigor in Memory-Augmented Cooperative MARL: A Study with CHARM

Cooperative multi-agent reinforcement learning (MARL) is hard when rewards are sparse. This is an evaluation study built around a negative result. We study CHARM (Cooperative Hindsight-Augmented Role-conditioned Memory), a four-part memory layer that wraps a standard backbone: role-indexed episodic memory (JACM/RCMP), hindsight relabeling (HCR), an auxiliary memory loss (SMAL), and cooperative replay (CER). We test it on four backbones (DQN, QMIX, MADDPG, QPLEX) against the episodic-memory baselines EMC and EMU, five to ten seeds, on Google Research Football. The mean win count barely moves on any backbone, and the small five-seed differences reverse sign at ten seeds. Because every method trains under reward shaping, the task is not fully sparse; a shaping-free control and a 30,000-episode anchor keep the result flat. A random-action policy scores within the trained range; only two of eleven configurations exceed it before correction. Inter-seed variance drops with CHARM on all four, but this is not a memory effect: plain weight decay reproduces the reduction, and removing the recall path leaves training byte-identical, so the cause is generic regularization. Evaluation is fragile: a three-seed comparison (g=2.07) collapsed at five seeds, and no comparison survived correction. A three-tier memory added nothing: its top tier was never consumed.