A Multi-Agent Reinforcement Learning Framework for Autonomous Traffic-Light-Free Intersection Management
Main Article Content
Abstract
Traffic signal control remains one of the most persistent sources of delay, fuel consumption, and inefficiency in urban road networks. Even adaptive signal systems rely on predefined phases that fail to exploit real-time vehicle-level intelligence. With the rapid emergence of connected and autonomous vehicles (CAVs) and vehicle-to-everything (V2X) communication, traffic-light-free intersections have become a viable alternative. This paper presents a novel Multi-Agent Reinforcement Learning (MARL) framework that enables vehicles to autonomously negotiate intersection passage without traffic lights. Each vehicle operates as an intelligent agent, coordinating with others through V2X communication while a lightweight Intersection Coordination Server (ICS) enforces safety constraints. A Graph Attention Network (GAT) captures dynamic spatial interactions among conflicting vehicles, and Multi-Agent Proximal Policy Optimization (MAPPO) ensures stable cooperative learning under partial observability. Extensive simulations conducted in Simulation of Urban Mobility (SUMO) and Car Learning to Act (CARLA) demonstrate substantial performance improvements, including up to 76% reduction in average delay, 48% increase in throughput, and up to 41% reduction in fuel consumption compared to adaptive signalized intersections. Results indicate that MARL-based, signal-free intersection control offers a scalable and safe pathway toward next-generation smart mobility.
Article Details
References
D. Schrank, B. Eisele, T. Lomax, and J. Bak, “2019 Urban Mobility Report,” Texas A&M Transportation Institute, College Station, TX, USA, Tech. Rep., 2019.
M. Barth and K. Boriboonsomsin, “Traffic congestion and greenhouse gases,” Access Magazine, no. 35, pp. 2–9, 2009.
P. Lowrie, “The Sydney coordinated adaptive traffic system—Principles, methodology, algorithms,” in Proc. IEE Int. Conf. Road Traffic Signalling, London, U.K., 1982, pp. 67–70.
S. E. Shladover, “Connected and automated vehicle systems: Introduction and overview,” J. Intell. Transp. Syst., vol. 22, no. 3, pp. 190–200, 2018.
H. Hartenstein and K. Laberteaux, “VANET: Vehicular Applications and Inter-Networking Technologies”, Hoboken, NJ, USA: Wiley, 2010.
K. Dresner and P. Stone, “A multiagent approach to autonomous intersection management,” J. Artif. Intell. Res., vol. 31, pp. 591–656, 2008.
K. Dresner and P. Stone, “Multiagent traffic management: An improved intersection control mechanism,” in Proc. 4th Int. Joint Conf. Autonomous Agents and Multiagent Systems, 2005, pp. 471–477.
A. Colombo and D. Del Vecchio, “Efficient algorithms for collision avoidance at intersections,” in Proc. 15th ACM Int. Conf. Hybrid Systems: Computation and Control, 2012, pp. 145–154.
E. Wei et al., “PressLight: Learning max pressure control to coordinate traffic signals in arterial network,” in Proc. 25th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, 2019, pp. 1290–1298.
T. Chu, J. Wang, L. Codecà, and Z. Li, “Multi-agent deep reinforcement learning for large-scale traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 3, pp. 1086–1095, Mar. 2020.
L. Li, Y. Lv, and F.-Y. Wang, “Traffic signal timing via deep reinforcement learning,” IEEE/CAA J. Autom. Sinica, vol. 3, no. 3, pp. 247–254, Jul. 2016.
L. Buşoniu, R. Babuška, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” IEEE Trans. Syst., Man, Cybern. C, vol. 38, no. 2, pp. 156–172, Mar. 2008.
R. Lowe et al., “Multi-agent actor-critic for mixed cooperative-competitive environments,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 6379–6390.
T. Rashid et al., “QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2018, pp. 4295–4304.
C. Yu et al., “The surprising effectiveness of PPO in cooperative multi-agent games,” arXiv:2103.01955, 2021.
P. Veličković et al., “Graph attention networks,” in Proc. Int. Conf. Learn. Representations (ICLR), 2018.
P. A. Lopez et al., “Microscopic traffic simulation using SUMO,” in Proc. IEEE Int. Conf. Intelligent Transportation Systems, 2018, pp. 2575–2582.
A. Dosovitskiy et al., “CARLA: An open urban driving simulator,” in Proc. 1st Annu. Conf. Robot Learning (CoRL), 2017, pp. 1–16.
M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,” Phys. Rev. E, vol. 62, no. 2, pp. 1805–1824, 2000.