Multiple-objective control of stormwater basins using deep reinforcement learning

ABSTRACT

Stormwater basins are important stormwater control measures which can reduce peak flow rate, mitigate flooding volume, and improve water quality during heavy rainfall events. Previous control strategies for stormwater basins have typically treated water quality and quantity as separate objectives. With the increasing urban runoff caused by climate change and urbanization, current single-objective control strategies cannot fully harness the control potential of basins and therefore require improvement. However, designing multi-objective control strategies for basins is challenging because of the conflicting operation goals and the complexity of the dynamic environmental conditions. This research proposes a novel real-time control strategy based on deep reinforcement learning to address these challenges. It employs a deep Q-network to develop an agent capable of making control decision. After being trained on three different rainfall events, the reinforcement learning agent can make appropriate decisions for previously unseen rainfall events. Compared to other two rule-based control scenarios and a static state scenario, the deep reinforcement learning method is more effective in terms of reducing total suspended solids, reducing peak flow, minimizing outflow flashiness, and controlling effort, striking a good balance between conflicting control objectives.