Ramp Metering for Multi‐Bottleneck Freeways With an Aggregate Modelling Framework: A Data‐Driven Approach Versus Model‐Based Methods

ABSTRACT

This paper develops a reinforcement learning (RL) approach for ramp metering, based on a parsimonious aggregate modelling framework that captures complex interactions among multiple bottlenecks in large‐scale freeways. This framework, represented by the macroscopic fundamental diagram (MFD), may provide a simple and robust representation for freeways but has not been sufficiently studied for controller development. This paper contributes to extending MFD‐based control in multi‐bottleneck freeways through the construction of a model‐free RL controller that addresses inherent mismatch between deterministic models and stochastic traffic dynamics. The state space is elaborately defined based on the MFD, considering density heterogeneity. The solution strategically adopts a prioritised experience replay (PER)‐based duelling double deep Q‐network (D3QN), guided by the shape of the MFD near dynamic critical accumulations. Subsequently, the computed total flow of all on‐ramps is distributed among individual on‐ramps using a classic method, based on local state feedback and the fundamental diagram. Finally, the proposed data‐driven approach is thoroughly compared with state‐of‐the‐art MFD model‐based methods and the HERO, under different demand fluctuations in microscopic simulations. Results indicate that MFD‐based controllers generally outperform the HERO, while their performance critically depends on their solution algorithms. Among them, the PER‐D3QN controller achieves the optimal overall traffic efficiency.