Reinforcement-Learning-Guided Two-Stage Multi-Energy Optimization for Carbon-Aware Charging Infrastructure

Electric vehicle (EV) parking lots can create concentrated charging demand that couples operating cost, grid capacity use, and carbon emissions. This paper proposes a reinforcement-learning-guided two-stage multi-energy optimization framework for carbon-aware EV parking lot charging. The reinforcement learning (RL) layer uses a Deep Q-Network (DQN) to generate a data-driven charging reference from state-of-charge (SoC), time-to-departure, vehicle presence, electricity price, and grid-load information. This profile is transferred to a two-stage optimization model as a behavioral reference rather than being used as the final dispatch schedule. The first stage determines baseline operation and residual grid headroom, while the second stage schedules EV charging together with Power-to-Gas (P2G) and Carbon Capture and Storage (CCS) decisions under capacity, carbon-budget, and multi-energy constraints. A soft-tracking formulation links the learned profile with the optimized schedule and allows the tracking coefficient to shape different operating regimes. The results show that the proposed framework improves grid feasibility, reduces peak charging stress, and enhances carbon-aware operation. CCS mainly supports carbon-budget feasibility, whereas P2G provides additional value when renewable surplus is available.