Replacing Tunable Parameters in Weather and Climate Models With State‐Dependent Functions Using Reinforcement Learning

Abstract

Weather and climate models rely on parameterizations to represent unresolved sub‐grid processes. Traditional schemes rely on fixed coefficients that are weakly constrained and tuned offline, contributing to persistent biases that limit their ability to adapt to underlying physics. This study presents a framework that learns components of parametrization schemes online as a function of the evolving model state using reinforcement learning (RL) and evaluates policy‐driven parameter updates across idealized testbeds spanning a simple climate bias correction (SCBC), a radiative‐convective equilibrium (RCE), and a zonal mean energy balance model (EBM) with single‐agent and federated multi‐agent settings. Across nine RL algorithms, Truncated Quantile Critics (TQC), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed DDPG (TD3) achieved the highest skill and stable convergence, with performance assessed against a static baseline using area‐weighted RMSE, temperature and pressure‐level diagnostics. For the EBM, single‐agent RL outperformed static parameter tuning with the strongest gains in tropical and mid‐latitude bands, while federated RL on multi‐agent setups enabled specialized control and faster convergence, with a six‐agent DDPG configuration using frequent aggregation yielding the lowest area‐weighted RMSE across the tropics and mid‐latitudes. The learned corrections were also physically meaningful as agents modulated EBM radiative parameters to reduce meridional biases, adjusted RCE lapse rates to match vertical temperature errors, and stabilized heating increments to limit drift. Overall, results show that RL can learn skillful state‐dependent parametrization components in idealized settings, offering a scalable pathway for online learning within numerical models and a starting point for evaluation in weather and climate models.