IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control

  title={IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control},
  author={Franois-Xavier Devailly and Denis Larocque and Laurent Charlin},
Scaling adaptive traffic-signal control involves dealing with combinatorial state and action spaces. Multi-agent reinforcement learning attempts to address this challenge by distributing control to specialized agents. However, specialization hinders generalization and transferability, and the computational graphs underlying neural-networks architectures---dominating in the multi-agent setting---do not offer the flexibility to handle an arbitrary number of entities which changes both between… Expand

Figures and Tables from this paper

Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations
This work proposes a new cooperative learning framework with pre-trained heterogeneous observation representations that can significantly outperform existing MARL baselines as well as operational research solutions that are widely-used in industry. Expand
IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal control
Experimental results on different traffic datasets illustrate that IHG-MA algorithm outperforms the state-of-the-art algorithms in terms of multiple traffic metrics, which seems to be a new promising algorithm for multi-intersection traffic signal control. Expand
Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learning
The overall research outcome suggests that by decomposing large networks using fog-nodes, the proposed fog-based graphic RL (FG-RL) model can be easily applied to scale into larger traffic networks. Expand
Cooperative Control for Multi-Intersection Traffic Signal Based on Deep Reinforcement Learning and Imitation Learning
A novel end-to-end learning model is established and an efficient training method is proposed analogously, which is capable of adapting to large-scale scenarios, and the input traffic status in multi-intersection are expressed by a tensor without information loss. Expand


Microscopic Traffic Simulation using SUMO
The latest developments concerning intermodal traffic solutions, simulator coupling and model development and validation on the example of the open source traffic simulator SUMO are presented. Expand
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control
This paper trains a deep recurrent controller that can automatically determine which actions move the end-effector of a robotic arm to a desired object and describes how the resulting model can be transferred to a real-world robot by disentangling perception from control and only adapting the visual layers. Expand
Dueling Network Architectures for Deep Reinforcement Learning
This paper presents a new neural network architecture for model-free reinforcement learning that leads to better policy evaluation in the presence of many similar-valued actions and enables the RL agent to outperform the state-of-the-art on the Atari 2600 domain. Expand
Double Q-learning
An alternative way to approximate the maximum expected value for any set of random variables is introduced and the obtained double estimator method is shown to sometimes underestimate rather than overestimate themaximum expected value. Expand
Large-Scale Traffic Signal Control Using a Novel Multiagent Reinforcement Learning
Co-DQL outperforms the state-of-the-art decentralized MARL algorithms in terms of multiple traffic metrics and is applied to TSC and tested on various traffic flow scenarios of TSC simulators. Expand
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
This paper presents a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent, advantage actor critic (A2C), within the context of ATSC, and presents two methods to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. Expand
Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control
This paper tackles the problem of multi-intersection traffic signal control, especially for large-scale networks, based on RL techniques and transportation theories and shows that implicit coordination could be achieved by individual control agents with well-crafted reward design thus reducing the dimensionality. Expand
A Deep Reinforcement Learning Network for Traffic Light Cycle Control
A deep reinforcement learning model is proposed to control the traffic light cycle that incorporates multiple optimization elements to improve the performance, such as dueling network, target network, double Q-learning network, and prioritized experience replay. Expand
CoLight: Learning Network-level Cooperation for Traffic Signal Control
The proposed model, CoLight, is the first to use graph attentional networks in the setting of reinforcement learning for traffic signal control and to conduct experiments on the large-scale road network with hundreds of traffic signals. Expand
Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
This paper surveys recent works that address the non-stationarity problem in multi-agent deep reinforcement learning, and methods range from modifications in the training procedure, to learning representations of the opponent's policy, meta-learning, communication, and decentralized learning. Expand