IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control

  title={IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control},
  author={Franois-Xavier Devailly and Denis Larocque and Laurent Charlin},
Scaling adaptive traffic-signal control involves dealing with combinatorial state and action spaces. Multi-agent reinforcement learning attempts to address this challenge by distributing control to specialized agents. However, specialization hinders generalization and transferability, and the computational graphs underlying neural-networks architectures---dominating in the multi-agent setting---do not offer the flexibility to handle an arbitrary number of entities which changes both between… Expand

Figures and Tables from this paper

Cooperative Control for Multi-Intersection Traffic Signal Based on Deep Reinforcement Learning and Imitation Learning
A novel end-to-end learning model is established and an efficient training method is proposed analogously, which is capable of adapting to large-scale scenarios, and the input traffic status in multi-intersection are expressed by a tensor without information loss. Expand
Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations
This work proposes a new cooperative learning framework with pre-trained heterogeneous observation representations that can significantly outperform existing MARL baselines as well as operational research solutions that are widely-used in industry. Expand


Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
This paper presents a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent, advantage actor critic (A2C), within the context of ATSC, and presents two methods to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. Expand
Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control
This paper tackles the problem of multi-intersection traffic signal control, especially for large-scale networks, based on RL techniques and transportation theories and shows that implicit coordination could be achieved by individual control agents with well-crafted reward design thus reducing the dimensionality. Expand
CoLight: Learning Network-level Cooperation for Traffic Signal Control
The proposed model, CoLight, is the first to use graph attentional networks in the setting of reinforcement learning for traffic signal control and to conduct experiments on the large-scale road network with hundreds of traffic signals. Expand
Multiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs
This paper presents the first application ofmax-plus to a large-scale problem and verifies its efficacy in realistic settings and provides empirical evidence that max-plus performs well on cyclic graphs, though it has been proven to converge only for tree-structured graphs. Expand
Reinforcement Learning With Function Approximation for Traffic Signal Control
A reinforcement learning (RL) algorithm with function approximation for traffic signal control that incorporates state-action features and is easily implementable in high-dimensional settings and outperforms all the other algorithms on all the road network settings that it considers. Expand
Using a Deep Reinforcement Learning Agent for Traffic Signal Control
This work applies modern deep reinforcement learning methods to build a truly adaptive traffic signal control agent in the traffic microsimulator SUMO, using a new state space, the discrete traffic state encoding, which is information dense. Expand
Multi-agent reinforcement learning for traffic signal control
This paper forms the TSC problem as a discounted cost Markov decision process (MDP) and applies multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies and shows that these algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm over two real road networks. Expand
STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Traffic Light Control
A novel SpatioTemporal Multi-Agent Reinforcement Learning (STMARL) framework for effectively capturing the spatio-temporal dependency of multiple related traffic lights and control these traffic lights in a coordinating way is proposed. Expand
Reinforcement learning for true adaptive traffic signal control
The ability to exert real-time, adaptive control of transportation processes is the core of many intelligent transportation systems decision support tools. Reinforcement learning, an artificialExpand
Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events
Abstract The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part ofExpand