An Introduction to Deep Reinforcement Learning

@article{FranoisLavet2018AnIT,
  title={An Introduction to Deep Reinforcement Learning},
  author={Vincent François-Lavet and Peter Henderson and Riashat Islam and Marc G. Bellemare and Joelle Pineau},
  journal={Found. Trends Mach. Learn.},
  year={2018},
  volume={11},
  pages={219-354}
}
Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. This field of research has been able to solve a wide range of complex decision-making tasks that were previously out of reach for a machine. Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more. This manuscript provides an introduction to deep reinforcement learning models, algorithms and techniques. Particular focus is on the… Expand

Paper Mentions

A Comprehensive Discussion on Deep Reinforcement Learning
Deep reinforcement learning (RL) is an important method to provide solutions for the perception and decision-making problems of complex systems. It has provided great contributions in naturalExpand
Deep Reinforcement Learning Techniques in Diversified Domains: A Survey
TLDR
It is found that even after obtaining good results in Atari, Go, Robotics, multi-agent scenarios, there are challenges such as generalization, satisfying multiple objectives, divergence, learning robust policy. Expand
A survey and critique of multiagent deep reinforcement learning
TLDR
A clear overview of current multiagent deep reinforcement learning (MDRL) literature is provided to help unify and motivate future research to take advantage of the abundant literature that exists in a joint effort to promote fruitful research in the multiagent community. Expand
Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review
TLDR
This paper presents recent significant progress of deep reinforcement learning algorithms, which try to tackle the problems for the application in the domain of robotic manipulation control, such as sample efficiency and generalization. Expand
Comparison of Multiple Reinforcement Learning and Deep Reinforcement Learning Methods for the Task Aimed at Achieving the Goal
Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) methods are a promising approach to solving complex tasks in the real world with physical robots. In this paper, we compare severalExpand
Deep Reinforcement Learning: A State-of-the-Art Walkthrough
TLDR
The key differences of the various kinds of algorithms are discussed, indicate their potential and limitations, as well as provide insights to researchers regarding future directions of the field. Expand
A Very Condensed Survey and Critique of Multiagent Deep Reinforcement Learning
TLDR
The primary goal of this extended abstract is to provide a broad overview of current multiagent deep reinforcement learning (MDRL) literature, hopefully motivating the reader to review the authors' 47page JAAMAS survey article [28]. Expand
State-of-the-Art Reinforcement Learning Algorithms
TLDR
This research paper brings together many different aspects of the current research on several fields associated to Reinforcement Learning, providing a wide variety of learning algorithms like Markov Decision Processes, Q Learning, Temporal Difference Learning, Actor-Critic Algorithms, Deep Deterministic Policy Gradients, Evolution Strategies Algorithm. Expand
Terminal Prediction as an Auxiliary Task for Deep Reinforcement Learning
TLDR
This paper contributes a novel self-supervised auxiliary task, i.e., Terminal Prediction (TP), estimating temporal closeness to terminal states for episodic tasks, to help representation learning by letting the agent predict how close it is to a terminal state, while learning its control policy. Expand
Transformer Based Reinforcement Learning For Games
TLDR
This work proposes a method utilizing a transformer network which have recently replaced RNNs in Natural Language Processing (NLP), and performs experiments to compare with existing methods. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 359 REFERENCES
Learning to reinforcement learn
TLDR
This work introduces a novel approach to deep meta-reinforcement learning, which is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure. Expand
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
TLDR
This paper proposes to represent a "fast" reinforcement learning algorithm as a recurrent neural network (RNN) and learn it from data, encoded in the weights of the RNN, which are learned slowly through a general-purpose ("slow") RL algorithm. Expand
Stochastic Neural Networks for Hierarchical Reinforcement Learning
TLDR
This work proposes a general framework that first learns useful skills in a pre-training environment, and then leverages the acquired skills for learning faster in downstream tasks, and uses Stochastic Neural Networks combined with an information-theoretic regularizer to efficiently pre-train a large span of skills. Expand
How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies
TLDR
When the discount factor progressively increases up to its final value, it is empirically shown that it is possible to significantly reduce the number of learning steps and the possibility to fall within a local optimum during the learning process, thus connecting the discussion with the exploration/exploitation dilemma. Expand
Continuous Deep Q-Learning with Model-based Acceleration
TLDR
This paper derives a continuous variant of the Q-learning algorithm, which it is called normalized advantage functions (NAF), as an alternative to the more commonly used policy gradient and actor-critic methods, and substantially improves performance on a set of simulated robotic control tasks. Expand
A Study on Overfitting in Deep Reinforcement Learning
TLDR
This paper conducts a systematic study of standard RL agents and finds that they could overfit in various ways and calls for more principled and careful evaluation protocols in RL. Expand
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
TLDR
It is demonstrated that a recent deep reinforcement learning algorithm based on off-policy training of deep Q-functions can scale to complex 3D manipulation tasks and can learn deep neural network policies efficiently enough to train on real physical robots. Expand
Recurrent Reinforcement Learning: A Hybrid Approach
TLDR
This work investigates a deep-learning approach to learning the representation of states in partially observable tasks, with minimal prior knowledge of the domain, and proposes a new family of hybrid models that combines the strength of both supervised learning and reinforcement learning, trained in a joint fashion. Expand
Asynchronous Methods for Deep Reinforcement Learning
TLDR
A conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers and shows that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input. Expand
Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
TLDR
This work defines a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains, and uses Atari games as a testing environment to demonstrate these methods. Expand
...
1
2
3
4
5
...