# Reinforcement learning in robotics: A survey

@article{Kober2013ReinforcementLI, title={Reinforcement learning in robotics: A survey}, author={J. Kober and J. Bagnell and Jan Peters}, journal={The International Journal of Robotics Research}, year={2013}, volume={32}, pages={1238 - 1274} }

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a… Expand

#### Supplemental Presentations

#### Figures, Tables, and Topics from this paper

#### Paper Mentions

#### 1,823 Citations

Survey of Model-Based Reinforcement Learning: Applications on Robotics

- Computer Science
- J. Intell. Robotic Syst.
- 2017

It is argued that, by employing model-based reinforcement learning, the—now limited—adaptability characteristics of robotic systems can be expanded, and model- based reinforcement learning exhibits advantages that makes it more applicable to real life use-cases compared to model-free methods. Expand

On decomposability in robot reinforcement learning

- Engineering
- 2017

Reinforcement learning is a computational framework that enables machines to learn from trialand-error interaction with the environment. In recent years, reinforcement learning has been successfully… Expand

Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

- Computer Science, Engineering
- ArXiv
- 2021

The last half-decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities.… Expand

An Algorithmic Perspective on Imitation Learning

- Computer Science
- Found. Trends Robotics
- 2018

This work provides an introduction to imitation learning, dividing imitation learning into directly replicating desired behavior and learning the hidden objectives of the desired behavior from demonstrations (called inverse optimal control or inverse reinforcement learning [Russell, 1998]). Expand

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

- Computer Science
- 2020 IEEE Symposium Series on Computational Intelligence (SSCI)
- 2020

The fundamental background behind sim-to-real transfer in deep reinforcement learning is covered and the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation are overviewed. Expand

Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system

- Computer Science, Engineering
- 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR)
- 2018

It is shown that control policies learned in simulation can successfully transfer to a physical system, composed of three Phantom robots pushing an object to various desired target positions, using a modified form of the natural policy gradient algorithm applied to a carefully identified simulation model. Expand

Robot Learning from Demonstration: A Review of Recent Advances

- 2019

In the context of robotics and automation, learning from demonstrations (LfD) is the paradigm in which robots acquire new skills by learning to imitate an expert. The choice of LfD over other robot… Expand

Comparison of deep reinforcement learning algorithms: Path Search in Grid World

- Computer Science
- 2021 International Conference on Electronics, Information, and Communication (ICEIC)
- 2021

This paper compares the simulation results of various deep reinforcement learning algorithms for path search, which is a representative problem depicted as a grid world, and presents the environment, model architecture and parameters used in the simulation. Expand

Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review

- Computer Science
- Robotics
- 2021

This paper presents recent significant progress of deep reinforcement learning algorithms, which try to tackle the problems for the application in the domain of robotic manipulation control, such as sample efficiency and generalization. Expand

On Training Flexible Robots using Deep Reinforcement Learning

- Computer Science
- 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- 2019

This paper systematically study the efficacy of policy search methods using DRL in training flexible robots and indicates that DRL is successfully able to learn efficient and robust policies for complex tasks at various degrees of flexibility. Expand

#### References

SHOWING 1-10 OF 251 REFERENCES

Generalized model learning for Reinforcement Learning on a humanoid robot

- Computer Science, Engineering
- 2010 IEEE International Conference on Robotics and Automation
- 2010

An algorithm, Reinforcement Learning with Decision Trees (RL-DT), that uses decision trees to learn the model by generalizing the relative effect of actions across states, and which is effective on an Aldebaran Nao humanoid robot scoring goals in a penalty kick scenario. Expand

A Framework for Reinforcement Learning on Real Robots

- Computer Science
- AAAI/IAAI
- 1998

One promising approach to learning on real robots that is attracting considerable interest at the moment is reinforcement learning, where the programmer must supply a reward function which maps states of the world onto a scalar reward, essentially saying how good or bad it is to be in a given state. Expand

Reinforcement learning of motor skills in high dimensions: A path integral approach

- Computer Science
- 2010 IEEE International Conference on Robotics and Automation
- 2010

This paper derives a novel approach to RL for parameterized control policies based on the framework of stochastic optimal control with path integrals, and believes that this new algorithm, Policy Improvement with Path Integrals (PI2), offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL in robotics. Expand

Module Based Reinforcement Learning: An Application to a Real Robot

- Computer Science, Engineering
- EWLR
- 1997

A systematic design method whose motivation comes from the desire to transform the task-to-be-solved into a finite-state, discrete-time, “approximately” Markovian task, which is completely observable too is suggested. Expand

On the sample complexity of reinforcement learning.

- Computer Science
- 2003

Novel algorithms with more restricted guarantees are suggested whose sample complexities are again independent of the size of the state space and depend linearly on the complexity of the policy class, but have only a polynomial dependence on the horizon time. Expand

Fast Reinforcement Learning for Vision-guided Mobile Robots

- Engineering, Computer Science
- Proceedings of the 2005 IEEE International Conference on Robotics and Automation
- 2005

This paper presents a new reinforcement learning algorithm for accelerating acquisition of new skills by real mobile robots, without requiring simulation. It speeds up Q-learning by applying… Expand

Policy search for motor primitives in robotics

- Computer Science
- Machine Learning
- 2010

A novel EM-inspired algorithm for policy learning that is particularly well-suited for dynamical system motor primitives is introduced and applied in the context of motor learning and can learn a complex Ball-in-a-Cup task on a real Barrett WAM™ robot arm. Expand

Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning

- Computer Science
- Robotics: Science and Systems
- 2011

It is demonstrated how a low-cost off-the-shelf robotic system can learn closed-loop policies for a stacking task in only a handful of trials-from scratch. Expand

Theory and application of reward shaping in reinforcement learning

- Psychology
- 2004

Applying conventional reinforcement to complex domains requires the use of an overly simplified task model, or a large amount of training experience. This problem results from the need to experience… Expand

Learning Generalizable Control Programs

- Computer Science
- IEEE Transactions on Autonomous Mental Development
- 2011

This paper demonstrates how the proposed intrinsic motivator for affordance discovery can cause a robot to both acquire such hierarchical policies using reinforcement learning and then to generalize these policies to new contexts. Expand