A General Reinforcement Learning Framework for Remedial Action Schemes Against False Data Injection Cyberattacks in Renewable-Integrated Power Systems: From Stochastic Optimization to Markov Decision Processes

The increasing penetration of renewable-based distributed generation (RDG) has significantly increased the operational uncertainty and complexity of modern power systems, making conventional optimization-based remedial action schemes (RASs) increasingly computationally demanding and less suitable for real-time critical applications. This paper proposes a general reinforcement learning (RL) framework for solving multi-period stochastic power system optimization problems, with RAS serving as the primary application against false data injection (FDI) cyberattacks targeting smart power systems. The proposed methodology consists of three sequential stages. First, the RAS is formulated as a stochastic nonlinear optimization model that minimizes the total operating cost while explicitly accounting for thermal generation costs, renewable generation costs, renewable power mismatch penalties, and energy storage operation, subject to power flow, operational, and network security constraints. Second, the optimization problem is systematically reformulated as a Markov decision process (MDP), where the system operating conditions define the state space, control variables constitute the action space, and a reward function is designed to capture both economic objectives and operational constraint violations. Finally, the resulting MDP is solved using an actor-critic RL algorithm that learns an optimal control policy directly from historical operating data without requiring an explicit mathematical model of system dynamics. Unlike problem-specific RL formulations, the proposed framework provides a unified methodology for transforming a broad class of stochastic power system optimization problems, including RAS against FDI attacks, optimal power flow, feeder reconfiguration, and loss minimization, into RL-compatible formulations. The framework naturally accommodates the stochastic behavior of renewable generation and sequential decision-making over multiple time intervals while enabling scalable and adaptive control to remediate targeted power system and bring it back to the normal operation in a timely manner. The formulation establishes a generic foundation for applying modern RL techniques to uncertainty-aware power system operation and control, thereby facilitating real-time decision-making in renewable-rich electricity networks.