A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers

@article{Wang2018ARL,
  title={A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers},
  author={Pin Wang and Chingyao Chan and Arnaud de La Fortelle},
  journal={2018 IEEE Intelligent Vehicles Symposium (IV)},
  year={2018},
  pages={1379-1384}
}
Lane change is a crucial vehicle maneuver which needs coordination with surrounding vehicles. [] Key Method Particularly, we treated both state space and action space as continuous, and designed a Q-function approximator that has a closed-form greedy policy, which contributes to the computation efficiency of our deep Q-learning algorithm. Extensive simulations are conducted for training the algorithm, and the results illustrate that the Reinforcement Learning based vehicle agent is capable of learning a…
View on IEEE
arxiv.org
120 Citations
Highly Influential Citations
7
Background Citations
47
Methods Citations
36

Figures from this paper

120 Citations

Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning
TLDR
This study proposes an automated lane change strategy using proximal policy optimization-based deep reinforcement learning, which shows great advantages in learning efficiency while still maintaining stable performance and results demonstrate the lane change maneuvers can be efficiently learned and executed in a safe, smooth and efficient manner.
Meta Reinforcement Learning-Based Lane Change Strategy for Autonomous Vehicles
TLDR
A meta reinforcement learning (MRL) method to improve the agent's generalization capabilities to new environments and shows more stable and efficient adaptation capabilities and can achieve 100% successful rate and 0% collision rate with only a few steps of gradient update adaptation.
A Novel Dynamic Lane-Changing Trajectory Planning Model for Automated Vehicles Based on Reinforcement Learning
  • Ce Yu, A. Ni, Yifeng Tu
  • Computer Science, Engineering
    Journal of Advanced Transportation
  • 2022
TLDR
A lane-changing model using the deep deterministic policy gradient method, which can simultaneously control the lateral and longitudinal motions of the vehicle, that can reduce collision risk and has a positive effect on the average speed of overall traffic flow is developed.
Driving Decision and Control for Automated Lane Change Behavior based on Deep Reinforcement Learning
TLDR
This paper proposed a hierarchical reinforcement learning based architecture for decision making and control of lane changing situations and demonstrated the effectiveness of this framework in simulation, from both the decision-making and control layers.
Automated Vehicle Control at Freeway Lane-drops: a Deep Reinforcement Learning Approach
TLDR
The research findings demonstrate the promise of DQN RL in allowing the CAV to intelligently navigate, through the lane drop site, and the robustness of the RL agent was further tested on various scenarios different from the training case.
End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm
TLDR
Using a deep deterministic policy gradient (DDPG) algorithm, an end-to-end method for automated lane changing based on lidar data is proposed, and autonomous vehicles trained using the proposed method can learn an automated lane-changing policy while considering safety, comfort, and efficiency.
An Interactive Lane Change Decision Making Model With Deep Reinforcement Learning
TLDR
Experimental results demonstrate the capability of the proposed model to execute lane change maneuver with comfortable and safety reference trajectory at an appropriate time instance and traffic gap in various highway traffic scenarios.
Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated Driving
TLDR
A lane change system that combines deep reinforcement learning and vehicular communication is proposed that learns straight-line driving and collision avoidance actions without vehicle dynamics knowledge.
Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model
TLDR
A human decision-making model in reinforcement learning is incorporated to control AVs for safe and efficient operations and the number of collisions in training and testing is reduced to zero, while the training accuracy is not impinged.
Optimizing Trajectories for Highway Driving with Offline Reinforcement Learning
TLDR
This work proposes a Reinforcement Learning-based approach, which learns target trajectory parameters for fully autonomous driving on highways, and demonstrates that this offline trained agent learns to drive smoothly, achieving velocities as close as possible to the desired velocity, while outperforming the other agents.
...
...

References

SHOWING 1-10 OF 22 REFERENCES
A machine learning approach for personalized autonomous lane change initiation and control
We study an algorithm that allows a vehicle to autonomously change lanes in a safe but personalized fashion without the driver's explicit initiation (e.g. activating the turn signals). Lane change…
Deep Reinforcement Learning for Simulated Autonomous Vehicle Control
TLDR
The use of Deep Q-Learning to control a simulated car via reinforcement learning is investigated, and an agent is successfully able to train an agent to control the simulated car in JavaScript Racer in some respects.
End-to-End Deep Reinforcement Learning for Lane Keeping Assist
TLDR
The effect of some restricted conditions, put on the car during the learning phase, on the convergence time for finishing its learning phase is explained and the results demonstrate learning of autonomous maneuvering in a scenario of complex road curvatures and simple interaction with other vehicles.
Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge
  • Pin Wang, Chingyao Chan
  • Computer Science
    2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC)
  • 2017
TLDR
This work applies a Long Short-Term Memory (LSTM) architecture to model the interactive environment, from which an internal state containing historical driving information is conveyed to a Deep Q-Network (DQN), which takes the internal state as input and generates Q-values as output for action selection.
A hierarchical Model Predictive Control framework for on-road formation control of autonomous vehicles
TLDR
This paper presents an approach for the formation control of autonomous vehicles traversing along a multi-lane road with obstacles and traffic using a hierarchical Model Predictive Control approach.
Lane Change Algorithm for Autonomous Vehicles via Virtual Curvature Method
This paper addresses the lane changing problem of autonomous vehicles when there is no road infrastructure support. The autonomous vehicle should drive from the current lane to the adjacent lane in…
Model-Free reinforcement learning with continuous action in practice
TLDR
The actor-critic algorithm is applied to learn on a robotic platform with a fast sensorimotor cycle and constitutes an important step towards practical real-time learning control with continuous action.
A MODEL FOR THE STRUCTURE OF LANE-CHANGING DECISIONS
Continuous Deep Q-Learning with Model-based Acceleration
TLDR
This paper derives a continuous variant of the Q-learning algorithm, which it is called normalized advantage functions (NAF), as an alternative to the more commonly used policy gradient and actor-critic methods, and substantially improves performance on a set of simulated robotic control tasks.
Lane change and path planning of autonomous vehicles using GIS
TLDR
This study expects that GIS information could substitute several roles of a local sensor, in turn enhancing the autonomous driving of an autonomous vehicle.
...
...