A neural fictitious self‐play anti‐jamming strategy for secondary frequency control in microgrids with imperfect observations

Abstract

The microgrid secondary control systems are facing increasing threats of cyberattacks, due to the involvement of wireless communication infrastructures between microgrid control center (MGCC) and distributed generators (DGs). While most existing works focused on either the links from DGs to the MGCC (feedback channels) or the links from the MGCC to DGs (forward channels) under perfect observations, the two wireless links could be attacked simultaneously and are subject to non‐malicious environment perturbations besides cyberattacks. In this work, we propose a decentralized cyber‐defense method for securing the microgrid secondary frequency control against rational jamming attacks in both feedback and forward channels that also face imperfect observations. A multi‐player multi‐stage security game is formulated to model the interactions between attackers and defenders. In the formulated game, vulnerability metrics based on the observability Gramians and controllability Gramians of the microgrid are included in the payoff function to count the impact of cyber‐layer actions on microgrid physical performance. A neural fictitious self‐play (NFSP) anti‐jamming approach is developed for this game, simultaneously considering both cyber‐physical characteristics and imperfect observations. In the proposed approach, a deep Q‐learning network (DQN) and a supervised learning network are jointly built to learn the optimal defense policy under the worst‐case. The NFSP‐based solution is proved to be a Nash equilibrium (NE) based solution. Extensive comparisons are made, and the results verify that the proposed defense policy can effectively defend against rational bi‐directional jamming attacks in the microgrid secondary frequency control.