Policy search for motor primitives in robotics

@article{Kober2010PolicySF,
  title={Policy search for motor primitives in robotics},
  author={Jens Kober and Jan Peters},
  journal={Machine Learning},
  year={2010},
  volume={84},
  pages={171-203}
}
Many motor skills in humanoid robotics can be learned using parametrized motor primitives. While successful applications to date have been achieved with imitation learning, most of the interesting motor learning problems are high-dimensional reinforcement learning problems. These problems are often beyond the reach of current reinforcement learning methods. In this paper, we study parametrized policy search methods and apply these to benchmark problems of motor primitive learning in robotics… Expand
Policy Search for Motor Primitives
TLDR
An EM-inspired algorithm applicable to complex motor learning tasks is developed and it is shown that it can learn the complex Ball-in-a-Cup task using a real Barrett WAMTM robot arm. Expand
Reinforcement learning of motor skills using Policy Search and human corrective advice
TLDR
The results show that the proposed method not only converges to higher rewards when learning movement primitives, but also that the learning is sped up by a factor of 4–40 times, depending on the task. Expand
Learning motor skills: from algorithms to robot experiments
  • J. Kober
  • Computer Science
  • it Inf. Technol.
  • 2012
TLDR
It is shown how motor primitives can be employed to learn motor skills on three different levels, which contributes to the state of the art in reinforcement learning applied to robotics both in terms of novel algorithms and applications. Expand
Deep Reinforcement Learning for Robotic Manipulation - The state of the art
TLDR
This work embodies a survey of the most recent algorithms, architectures and their implementations in simulations and real world robotic platforms and manifests some of the state of the art applications of these approaches in robotic manipulation tasks. Expand
Learning Motor Skills - From Algorithms to Robot Experiments
TLDR
This book illustrates a method that learns to generalize parameterized motor plans which is obtained by imitation or reinforcement learning, by adapting a small set of global parameters and appropriate kernel-based reinforcement learning algorithms. Expand
Reinforcement learning to adjust parametrized motor primitives to new situations
TLDR
This paper proposes a method that learns to generalize parametrized motor plans by adapting a small set of global parameters, called meta-parameters, and introduces an appropriate reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. Expand
Deep predictive policy training using reinforcement learning
TLDR
A data-efficient deep predictive policy training (DPPT) framework with a deep neural network policy architecture which maps an image observation to a sequence of motor activations and is demonstrated by training predictive policies for skilled object grasping and ball throwing on a PR2 robot. Expand
Overcoming Exploration in Reinforcement Learning with Demonstrations
TLDR
This work uses demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm. Expand
Practical Learning Algorithms for Motor Primitives in Robotics
TLDR
Inspired by this example, it is shown how both single-stroke and rhythmic tasks can be learned efficiently by mimicking the human presenter with subsequent reward-driven self-improvement. Expand
Using reward-weighted imitation for robot Reinforcement Learning
  • Jan Peters, J. Kober
  • Computer Science
  • 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning
  • 2009
TLDR
This work generates a framework for policy learning which both unifies previous reinforcement learning approaches and allows the derivation of novel algorithms in the domain of anthropomorphic robotics. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 83 REFERENCES
Policy Search for Motor Primitives in Robotics
TLDR
This paper extends previous work on policy learning from the immediate reward case to episodic reinforcement learning, resulting in a general, common framework also connected to policy gradient methods and yielding a novel algorithm for policy learning that is particularly well-suited for dynamic motor primitives. Expand
Reinforcement Learning for Humanoid Robotics
TLDR
This paper discusses different approaches of reinforcement learning in terms of their applicability in humanoid robotics, and demonstrates that ‘vanilla’ policy gradient methods can be significantly improved using the natural policy gradient instead of the regular policy gradient. Expand
Policy Gradient Methods for Robotics
  • Jan Peters, S. Schaal
  • Engineering, Computer Science
  • 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems
  • 2006
TLDR
An overview on learning with policy gradient methods for robotics with a strong focus on recent advances in the field is given and how the most recently developed methods can significantly improve learning performance is shown. Expand
Reinforcement learning of motor skills in high dimensions: A path integral approach
TLDR
This paper derives a novel approach to RL for parameterized control policies based on the framework of stochastic optimal control with path integrals, and believes that this new algorithm, Policy Improvement with Path Integrals (PI2), offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL in robotics. Expand
Learning motor primitives for robotics
  • J. Kober, Jan Peters
  • Computer Science
  • 2009 IEEE International Conference on Robotics and Automation
  • 2009
TLDR
It is shown that two new motor skills, i.e., Ball-in-a-Cup and Ball-Paddling, can be learned on a real Barrett WAM robot arm at a pace similar to human learning while achieving a significantly more reliable final performance. Expand
Learning Attractor Landscapes for Learning Motor Primitives
TLDR
By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be generated without losing the stability properties of the canonical system. Expand
Learning perceptual coupling for motor primitives
TLDR
An augmented version of the dynamic system-based motor primitives which incorporates perceptual coupling to an external variable is proposed which can perform complex tasks such a Ball-in-a-Cup or Kendama task even with large variances in the initial conditions where a skilled human player would be challenged. Expand
Reinforcement learning by reward-weighted regression for operational space control
TLDR
This work uses a generalization of the EM-base reinforcement learning framework suggested by Dayan & Hinton to reduce the problem of learning with immediate rewards to a reward-weighted regression problem with an adaptive, integrated reward transformation for faster convergence. Expand
Machine Learning for motor skills in robotics
TLDR
This work investigates the ingredients for a general approach to motor skill learning and study two major components for such an approach, i.e., a theoretically well-founded general approach for representing the required control structures for task representation and execution and appropriate learning algorithms which can be applied in this setting. Expand
Towards Direct Policy Search Reinforcement Learning for Robot Control
TLDR
The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior and is demonstrated with simulated experiments using the underwater robot GARBI in a target reaching task. Expand
...
1
2
3
4
5
...