# An Introduction to Deep Reinforcement Learning

@article{FranoisLavet2018AnIT, title={An Introduction to Deep Reinforcement Learning}, author={Vincent François-Lavet and Peter Henderson and Riashat Islam and Marc G. Bellemare and Joelle Pineau}, journal={Found. Trends Mach. Learn.}, year={2018}, volume={11}, pages={219-354} }

Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. This field of research has been able to solve a wide range of complex decision-making tasks that were previously out of reach for a machine. Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more. This manuscript provides an introduction to deep reinforcement learning models, algorithms and techniques. Particular focus is on the… Expand

#### Supplemental Content

#### Figures, Tables, and Topics from this paper

#### Paper Mentions

#### 326 Citations

A Comprehensive Discussion on Deep Reinforcement Learning

- 2021 International Conference on Communications, Information System and Computer Engineering (CISCE)
- 2021

Deep reinforcement learning (RL) is an important method to provide solutions for the perception and decision-making problems of complex systems. It has provided great contributions in natural… Expand

Deep Reinforcement Learning Techniques in Diversified Domains: A Survey

- Computer Science
- 2021

It is found that even after obtaining good results in Atari, Go, Robotics, multi-agent scenarios, there are challenges such as generalization, satisfying multiple objectives, divergence, learning robust policy. Expand

A survey and critique of multiagent deep reinforcement learning

- Computer Science
- Autonomous Agents and Multi-Agent Systems
- 2019

A clear overview of current multiagent deep reinforcement learning (MDRL) literature is provided to help unify and motivate future research to take advantage of the abundant literature that exists in a joint effort to promote fruitful research in the multiagent community. Expand

Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review

- Computer Science
- Robotics
- 2021

This paper presents recent significant progress of deep reinforcement learning algorithms, which try to tackle the problems for the application in the domain of robotic manipulation control, such as sample efficiency and generalization. Expand

Comparison of Multiple Reinforcement Learning and Deep Reinforcement Learning Methods for the Task Aimed at Achieving the Goal

- 2021

Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) methods are a promising approach to solving complex tasks in the real world with physical robots. In this paper, we compare several… Expand

Deep Reinforcement Learning: A State-of-the-Art Walkthrough

- Computer Science
- J. Artif. Intell. Res.
- 2020

The key differences of the various kinds of algorithms are discussed, indicate their potential and limitations, as well as provide insights to researchers regarding future directions of the field. Expand

A Very Condensed Survey and Critique of Multiagent Deep Reinforcement Learning

- Computer Science
- AAMAS
- 2020

The primary goal of this extended abstract is to provide a broad overview of current multiagent deep reinforcement learning (MDRL) literature, hopefully motivating the reader to review the authors' 47page JAAMAS survey article [28]. Expand

State-of-the-Art Reinforcement Learning Algorithms

- Computer Science
- 2020

This research paper brings together many different aspects of the current research on several fields associated to Reinforcement Learning, providing a wide variety of learning algorithms like Markov Decision Processes, Q Learning, Temporal Difference Learning, Actor-Critic Algorithms, Deep Deterministic Policy Gradients, Evolution Strategies Algorithm. Expand

Terminal Prediction as an Auxiliary Task for Deep Reinforcement Learning

- Computer Science, Mathematics
- AIIDE
- 2019

This paper contributes a novel self-supervised auxiliary task, i.e., Terminal Prediction (TP), estimating temporal closeness to terminal states for episodic tasks, to help representation learning by letting the agent predict how close it is to a terminal state, while learning its control policy. Expand

Transformer Based Reinforcement Learning For Games

- Computer Science
- ArXiv
- 2019

This work proposes a method utilizing a transformer network which have recently replaced RNNs in Natural Language Processing (NLP), and performs experiments to compare with existing methods. Expand

#### References

SHOWING 1-10 OF 359 REFERENCES

Learning to reinforcement learn

- Computer Science, Psychology
- CogSci
- 2017

This work introduces a novel approach to deep meta-reinforcement learning, which is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure. Expand

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

- Computer Science, Mathematics
- ArXiv
- 2016

This paper proposes to represent a "fast" reinforcement learning algorithm as a recurrent neural network (RNN) and learn it from data, encoded in the weights of the RNN, which are learned slowly through a general-purpose ("slow") RL algorithm. Expand

Stochastic Neural Networks for Hierarchical Reinforcement Learning

- Computer Science
- ICLR
- 2017

This work proposes a general framework that first learns useful skills in a pre-training environment, and then leverages the acquired skills for learning faster in downstream tasks, and uses Stochastic Neural Networks combined with an information-theoretic regularizer to efficiently pre-train a large span of skills. Expand

How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies

- Computer Science
- ArXiv
- 2015

When the discount factor progressively increases up to its final value, it is empirically shown that it is possible to significantly reduce the number of learning steps and the possibility to fall within a local optimum during the learning process, thus connecting the discussion with the exploration/exploitation dilemma. Expand

Continuous Deep Q-Learning with Model-based Acceleration

- Computer Science
- ICML
- 2016

This paper derives a continuous variant of the Q-learning algorithm, which it is called normalized advantage functions (NAF), as an alternative to the more commonly used policy gradient and actor-critic methods, and substantially improves performance on a set of simulated robotic control tasks. Expand

A Study on Overfitting in Deep Reinforcement Learning

- Computer Science, Mathematics
- ArXiv
- 2018

This paper conducts a systematic study of standard RL agents and finds that they could overfit in various ways and calls for more principled and careful evaluation protocols in RL. Expand

Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates

- Computer Science
- 2017 IEEE International Conference on Robotics and Automation (ICRA)
- 2017

It is demonstrated that a recent deep reinforcement learning algorithm based on off-policy training of deep Q-functions can scale to complex 3D manipulation tasks and can learn deep neural network policies efficiently enough to train on real physical robots. Expand

Recurrent Reinforcement Learning: A Hybrid Approach

- Computer Science
- ArXiv
- 2015

This work investigates a deep-learning approach to learning the representation of states in partially observable tasks, with minimal prior knowledge of the domain, and proposes a new family of hybrid models that combines the strength of both supervised learning and reinforcement learning, trained in a joint fashion. Expand

Asynchronous Methods for Deep Reinforcement Learning

- Computer Science, Mathematics
- ICML
- 2016

A conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers and shows that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input. Expand

Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

- Computer Science, Mathematics
- ICLR
- 2016

This work defines a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains, and uses Atari games as a testing environment to demonstrate these methods. Expand