A Spatiotemporal-Awareness MARL Voltage Regulation Method for Distribution Networks with High Penetration Ratios

To tackle frequent voltage violations in distribution networks with high photovoltaic penetration ratios, voltage regulation methods that dispatch the active and reactive powers of PV panels are widely studied. Because of the costly global communication networks, in current distributed voltage regulation methods based on multi-agent reinforcement learning, each agent determines control actions according to its individual observation, which implies a partially observable Markov decision process. However, the determined control actions are far from optimal, resulting in less active power generated by PV panels and voltage violations. Some recent voltage regulation methods, where each agent determines control actions according to its historical data of the most recent time steps, can improve the optimality, but the improvement is still not enough. This paper proposes a spatiotemporal multi-agent reinforcement learning voltage regulation (ST-MARL-VR) method where each agent determines control actions according to its historical observations and those of the neighboring agents. Historical observations of neighboring agents are transferred through a point-to-point communication network, which can largely reduce the costs for communication network establishment and improve the communication robustness compared to global communication networks. Furthermore, in practice, our proposed method can be deployed on the terminal units in distribution networks. Subsequently, we validate the proposed ST-MARL-VR method under two test scenarios, and the quantitative performance improvements are above 20% compared with the two comparative methods. Comparative analyses against state-of-the-art benchmarks demonstrate that the developed ST-MARL-VR algorithm substantially enhances the optimality of the determined control actions.