CoopSECRM2D: A Safe, Efficient, and Comfortable Multi‐Agent Reinforcement Learning Model for Cooperative On‐Ramp Merging

ABSTRACT

Traffic congestion, especially at the on‐ramp merging bottleneck, has a significant negative impact on freeway performance, affecting travel efficiency, causing capacity reduction and deteriorating traveller safety and comfort. Within this context and with the development of connected and automated vehicle (CAV) technology, cooperative on‐ramp merging shows promise. In this research, we introduce the CoopSECRM2D, a novel cooperative multi‐agent reinforcement learning (MARL) framework designed to optimise on‐ramp merging for CAVs on freeways.

To be specific, the CoopSECRM2D model extends the RL‐based efficient and comfortable route‐following model (SECRM2D) to incorporate cooperative behaviours, encouraging vehicle agents to collaborate with surrounding agents. Additionally, this MARL framework introduces a dynamic scan range for efficient information sharing and improves local vehicle coordination using a novel cooperative reward mechanism. Meanwhile, the cooperation factors are also enhanced to be dynamically adjusted to mitigate speed differences between the vehicles in the main lane and ramp lane.

We conducted extensive modelling and evaluation in the Simulation of Urban MObility (SUMO) under various traffic demand scenarios to validate the effectiveness of the proposed CoopSECRM2D framework. Our experiments, spanning both synthetic setups and a real‐world highway network (Queen Elizabeth Way in Ontario, Canada), demonstrate that the cooperative model consistently outperforms baseline methods. Notably, the framework enhances micro‐level metrics such as longitudinal efficiency, comfort and mandatory lane‐changing behaviour, while also indirectly contributing to improved macro‐level performance, including better bottleneck outflow.