Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation

@article{Hoel2020TacticalDI,
  title={Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation},
  author={Carl-Johan Hoel and Krister Wolff and Leo Laine},
  journal={2020 IEEE Intelligent Vehicles Symposium (IV)},
  year={2020},
  pages={1563-1569}
}
Reinforcement learning (RL) can be used to create a tactical decision-making agent for autonomous driving. However, previous approaches only output decisions and do not provide information about the agent's confidence in the recommended actions. This paper investigates how a Bayesian RL technique, based on an ensemble of neural networks with additional randomized prior functions (RPF), can be used to estimate the uncertainty of decisions in autonomous driving. A method for classifying whether… 
Decision-Making Strategy on Highway for Autonomous Vehicles Using Deep Reinforcement Learning
TLDR
A deep reinforcement learning (DRL)-enabled decision-making policy is constructed for autonomous vehicles to address the overtaking behaviors on the highway to demonstrate the effectiveness of the DDQN-based overtaking policy.
Dueling Deep Q Network for Highway Decision Making in Autonomous Vehicles: A Case Study
TLDR
This work optimizes the highway decision making strategy of autonomous vehicles by using deep reinforcement learning (DRL) and reveals that the ego vehicle could safely and efficiently accomplish the driving task after learning and training.
A Novel Internal Resistance Curve Based State of Health Method to Estimate Battery Capacity Fade and Resistance Rise
TLDR
This work proposes a novel internal resistance curve-based state estimation method that estimates the battery capacity fade and resistance rise based on the curve of the internal resistance and thus provides better accuracy than the existing methods.
Reinforcement Learning with Uncertainty Estimation for Tactical Decision-Making in Intersections
TLDR
This work demonstrates one possible application of such a confidence estimate, by using this information to choose safe actions in unknown situations, which removes all collisions from within the training distribution, and most collisions outside of the distribution.
Energy optimization in autonomous driving using deep reinforced learning and stochastic system modelling
In the development of future low-emission vehicles, Machine Learning is playing a significant role, since manufacturers progressively hit constraints with existing technology. In addition to
Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving
TLDR
The results show that the EQN method can balance risk and time efficiency in different occluded intersection scenarios, by considering the estimated aleatoric uncertainty, and it is shown that the trained agent can use the epistemic uncertainty information to identify situations that the agent has not been trained for and thereby avoid making unfounded, potentially dangerous, decisions outside of the training distribution.
Learning Robot Swarm Tactics over Complex Adversarial Environments
TLDR
This paper presents a systematic approach to learn tactical mission-specific policies that compose primitives in a swarm to accomplish the mission efficiently using neural networks with special input and output encoding.
Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation
TLDR
This work presents the Attentiondriven Robotic Manipulation (ARM) algorithm, which is a general manipulation algorithm that can be applied to a range of sparse-rewarded tasks, given only a small number of demonstrations.
Sparsity Increases Uncertainty Estimation in Deep Ensemble
TLDR
It is empirically showed that the ensemble members’ disagreement increases with pruning, making models sparser by zeroing irrelevant parameters, which helps in making more robust predictions, and an energy-efficient compressed deep ensemble is appropriate for memory-intensive and uncertainty-aware tasks.
Transformer Uncertainty Estimation with Hierarchical Stochastic Attention
TLDR
This work proposes a novel way to enable transformers to have the capability of uncertainty estimation and retain the original predictive performance by learning a hierarchical stochastic self-attention that attends to values and a set of learnable centroids.
...
1
2
...

References

SHOWING 1-10 OF 42 REFERENCES
Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving
TLDR
A general framework for tactical decision making is introduced, which combines the concepts of planning and learning, in the form of Monte Carlo tree search and deep reinforcement learning, based on the AlphaGo Zero algorithm, extended to a domain with a continuous state space where self-play cannot be used.
Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
TLDR
This paper applies deep reinforcement learning to the problem of forming long term driving strategies and shows how policy gradient iterations can be used without Markovian assumptions, and decomposes the problem into a composition of a Policy for Desires and trajectory planning with hard constraints.
A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers
TLDR
This study proposed a Reinforcement Learning based approach to train the vehicle agent to learn an automated lane change behavior such that it can intelligently make a lane change under diverse and even unforeseen scenarios.
Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning
TLDR
A novel way of applying a convolutional neural network to high level input that represents interchangeable objects is introduced, and it is shown that the method produced an agent that matched or surpassed the performance of a commonly used reference model.
Bayesian Q-Learning
TLDR
This paper extends Watkins' Q-learning by maintaining and propagating probability distributions over the Q-values and establishes the convergence properties of the algorithm, which can exhibit substantial improvements over other well-known model-free exploration strategies.
Decision Making Under Uncertainty: Theory and Application
TLDR
This book provides an introduction to the challenges of decision making under uncertainty from a computational perspective and presents both the theory behind decision making models and algorithms and a collection of example applications that range from speech recognition to aircraft collision avoidance.
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
TLDR
The ChauffeurNet model can handle complex situations in simulation, and the perturbations then provide an important signal for these losses and lead to robustness of the learned model.
Virtual to Real Reinforcement Learning for Autonomous Driving
TLDR
A novel realistic translation network is proposed to make model trained in virtual environment be workable in real world, and is believed to be the first successful case of driving policy trained by reinforcement learning that can adapt to real world driving data.
Learning to Drive in a Day
TLDR
This work demonstrates a new framework for autonomous driving which moves away from reliance on defined logical rules, mapping, and direct supervision and provides a general and easy to obtain reward: the distance travelled by the vehicle without the safety driver taking control.
Continuous control with deep reinforcement learning
TLDR
This work presents an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces, and demonstrates that for many of the tasks the algorithm can learn policies end-to-end: directly from raw pixel inputs.
...
1
2
3
4
5
...