Reinforcement Learning-Based Dynamic Optimization Strategy of WAG Flooding with Inter-Well Connectivity for High Water-Cut Reservoirs

In high water-cut reservoirs, water flooding efficiency declines due to high-permeability channels that aggravate heterogeneity and cause water channeling. Gas flooding technology can improve recovery but is limited by operating conditions and costs. Water-alternating-gas (WAG) injection combines the advantages of both methods, but the complex reservoir connectivity and the strong inter-well interference hinder accurate displacement path control, which will reduce the oil production efficiency and increase costs. In this paper, a reinforcement learning-based dynamic optimization strategy is proposed to solve the injection optimization problem for WAG injection with inter-well connectivity. Specifically, a net present value (NPV) model is built for a single time slot based on the oil production, the injection cost of water or gas, the conversion cost between water flooding and gas flooding, and the inter-well connectivity. Furthermore, an NPV optimization problem is formulated for multi-injection wells and multi-production wells to enhance oil recovery. Then, a Q-learning algorithm is adopted for solving the proposed optimization problem to achieve the optimal WAG injection strategy, in which the state space comprises pressure and three-phase saturation, the action space includes injection type and rate, and the reward function is the total recovery benefit. Finally, extensive numerical simulations are conducted. Quantitative results demonstrate that the proposed algorithm achieves a converged NPV of approximately $8.3 million, outperforming the fixed-rate WAG strategy by 3.75%, the water flooding strategy by 9.21%, and the gas flooding strategy by 12.16%. Moreover, the proposed method increases cumulative oil production by 4.6% over fixed-rate WAG, 12.8% over water flooding, and 18.7% over gas flooding, while maintaining a competitive water cut of 0.734. The experimental results imply that the proposed WAG strategy can obtain higher NPVs than that of the benchmark algorithms.