Reinforcement Learning-Based Temperature Control of Lab-Scale Continuous Stirred Tank Reactor

Abstract

Continuous Stirred Tank Reactors (CSTRs) are widely used in chemical process industries, where precise temperature control is crucial for product quality, operational effectiveness, and safety. However, the accuracy of strict first-principle models is limited by the extremely nonlinear thermal behavior brought on by Arrhenius-based kinetics, measurement noise, transport delay, and feed fluctuations. In order to overcome this, the current study uses multivariate pilot-plant time-series data to create a high-fidelity nonlinear autoregressive model with exogenous inputs (NARX). Twin Delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC) are used in the NARX model as a reinforcement learning environment for closed-loop temperature regulation. Feed temperatures, feed flow rates, and hot-oil-bath actuation are among the various manipulated-variable combinations that are assessed. The suggested NARX–RL framework effectively tracks set points, according to the results. Both SAC and TD3 exhibit acceptable closed-loop performance, according to a comparative analysis; however, TD3 consistently outperforms SAC in tracking indices, convergence speed, and control stability across the majority of manipulated-variable combinations. Overall, the proposed NARX-based reinforcement learning technique provides a feasible option for flexible, data-driven, and practically implementable temperature regulation in lab-scale CSTR systems.