Receding-horizon neuro-fuzzy reinforcement learning with experience replay for containment control of multi-agent systems

Containment control for multi-agent systems (MAS) under disturbances is an important research topic. Although numerous containment control methods have been proposed, learning-based optimal control approaches for MAS still suffer from the drawbacks of low policy interpretability and poor convergence efficiency. In this paper, a receding-horizon neuro-fuzzy reinforcement learning approach with experience replay is proposed for the containment control of MAS. In the proposed method, the optimal control problem of MAS is formulated as a sequence of finite-horizon forward-receding optimization problems. Then, a value iteration framework that integrates policy improvement and policy optimization is adopted. Moreover, a distributed actor-critic policy framework based on fuzzy neural networks is designed and combined with a weight update mechanism using experience replay, thus improving online learning efficiency and enhancing policy interpretability. Finally, numerical simulations are conducted, and the results demonstrate that the proposed approach outperforms the receding-horizon reinforcement learning and neural network-based heuristic dynamic programming methods in terms of performance cost and mean error norm.