Policy search for motor primitives in robotics

@article{Kober2008PolicySF,
  title={Policy search for motor primitives in robotics},
  author={Jens Kober and Jan Peters},
  journal={Machine Learning},
  year={2008},
  volume={84},
  pages={171-203}
}
Many motor skills in humanoid robotics can be learned using parametrized motor primitives. While successful applications to date have been achieved with imitation learning, most of the interesting motor learning problems are high-dimensional reinforcement learning problems. These problems are often beyond the reach of current reinforcement learning methods. In this paper, we study parametrized policy search methods and apply these to benchmark problems of motor primitive learning in robotics… 

Policy Search for Motor Primitives

An EM-inspired algorithm applicable to complex motor learning tasks is developed and it is shown that it can learn the complex Ball-in-a-Cup task using a real Barrett WAMTM robot arm.

Imitation and Reinforcement Learning for Motor Primitives with Perceptual Coupling

This paper improves the open-loop policies and subsequently the perceptual coupling using a novel reinforcement learning method which is particularly well-suited for dynamical system-based motor primitives.

Reinforcement learning of motor skills using Policy Search and human corrective advice

The results show that the proposed method not only converges to higher rewards when learning movement primitives, but also that the learning is sped up by a factor of 4–40 times, depending on the task.

Learning motor skills: from algorithms to robot experiments

  • J. Kober
  • Computer Science
    it Inf. Technol.
  • 2012
It is shown how motor primitives can be employed to learn motor skills on three different levels, which contributes to the state of the art in reinforcement learning applied to robotics both in terms of novel algorithms and applications.

Learning New Basic Movements for Robotics

This work shows how both discrete and rhythmic tasks can be learned using a concerted approach of both imitation and reinforcement learning, and presents the current best performing learning algorithms.

Learning Motor Skills - From Algorithms to Robot Experiments

This book illustrates a method that learns to generalize parameterized motor plans which is obtained by imitation or reinforcement learning, by adapting a small set of global parameters and appropriate kernel-based reinforcement learning algorithms.

Learning motor primitives for robotics

  • J. KoberJan Peters
  • Computer Science
    2009 IEEE International Conference on Robotics and Automation
  • 2009
It is shown that two new motor skills, i.e., Ball-in-a-Cup and Ball-Paddling, can be learned on a real Barrett WAM robot arm at a pace similar to human learning while achieving a significantly more reliable final performance.

Reinforcement Learning to Adjust Robot Movements to New Situations

This paper describes how to learn such mappings from circumstances to meta-parameters using reinforcement learning, and uses a kernelized version of the reward-weighted regression to do so.

Learning Replanning Policies With Direct Policy Search

This work proposes a framework to learn trajectory replanning policies via contextual policy search and demonstrates that they are safe for the robot, can be learned efficiently, and outperform non-replanning policies for problems with partially observable or perturbed context.

Deep Reinforcement Learning for Robotic Manipulation - The state of the art

This work embodies a survey of the most recent algorithms, architectures and their implementations in simulations and real world robotic platforms and manifests some of the state of the art applications of these approaches in robotic manipulation tasks.
...

References

SHOWING 1-10 OF 85 REFERENCES

Reinforcement Learning for Humanoid Robotics

This paper discusses different approaches of reinforcement learning in terms of their applicability in humanoid robotics, and demonstrates that ‘vanilla’ policy gradient methods can be significantly improved using the natural policy gradient instead of the regular policy gradient.

Policy Gradient Methods for Robotics

  • Jan PetersS. Schaal
  • Computer Science
    2006 IEEE/RSJ International Conference on Intelligent Robots and Systems
  • 2006
An overview on learning with policy gradient methods for robotics with a strong focus on recent advances in the field is given and how the most recently developed methods can significantly improve learning performance is shown.

Learning motor primitives for robotics

  • J. KoberJan Peters
  • Computer Science
    2009 IEEE International Conference on Robotics and Automation
  • 2009
It is shown that two new motor skills, i.e., Ball-in-a-Cup and Ball-Paddling, can be learned on a real Barrett WAM robot arm at a pace similar to human learning while achieving a significantly more reliable final performance.

Learning Attractor Landscapes for Learning Motor Primitives

By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be generated without losing the stability properties of the canonical system.

Learning perceptual coupling for motor primitives

  • J. KoberB. MohlerJan Peters
  • Computer Science, Psychology
    2008 IEEE/RSJ International Conference on Intelligent Robots and Systems
  • 2008
An augmented version of the dynamic system-based motor primitives which incorporates perceptual coupling to an external variable is proposed which can perform complex tasks such a Ball-in-a-Cup or Kendama task even with large variances in the initial conditions where a skilled human player would be challenged.

Reinforcement learning by reward-weighted regression for operational space control

This work uses a generalization of the EM-base reinforcement learning framework suggested by Dayan & Hinton to reduce the problem of learning with immediate rewards to a reward-weighted regression problem with an adaptive, integrated reward transformation for faster convergence.

Towards Direct Policy Search Reinforcement Learning for Robot Control

The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior and is demonstrated with simulated experiments using the underwater robot GARBI in a target reaching task.

Teaching by Showing in Kendama Based on Optimization Principle

A general computational theory is proposed that derives representations for a wide variety of motor behaviors based on the dynamic optimization theory for trajectory formation that is applicable to higher-level task learning.

State-Dependent Exploration for Policy Gradient Methods

This work systematically derive SDE and applies it to several illustrative toy problems and a challenging robotics simulation task, where SDE greatly outperforms random exploration.

Robot motor skill coordination with EM-based Reinforcement Learning

An approach allowing a robot to acquire new motor skills by learning the couplings across motor control variables through Expectation-Maximization based Reinforcement Learning is presented.
...