Reinforcement learning of motor skills with policy gradients

@article{Peters2008ReinforcementLO,
  title={Reinforcement learning of motor skills with policy gradients},
  author={Jan Peters and S. Schaal},
  journal={Neural networks : the official journal of the International Neural Network Society},
  year={2008},
  volume={21 4},
  pages={
          682-97
        }
}
  • Jan Peters, S. Schaal
  • Published 2008
  • Computer Science, Medicine
  • Neural networks : the official journal of the International Neural Network Society
Autonomous learning is one of the hallmarks of human and animal behavior, and understanding the principles of learning will be crucial in order to achieve true autonomy in advanced machines like humanoid robots. In this paper, we examine learning of complex motor skills with human-like limbs. While supervised learning can offer useful tools for bootstrapping behavior, e.g., by learning from demonstration, it is only reinforcement learning that offers a general approach to the final trial-and… Expand
Reinforcement Learning for Motor Primitives
Humans demonstrate a large variety of very complicated motor skills in their day-to-day life. Their agility and adaptability to new control task remains unmatched by the millions of robots laboringExpand
Reinforcement learning of motor skills using Policy Search and human corrective advice
TLDR
The results show that the proposed method not only converges to higher rewards when learning movement primitives, but also that the learning is sped up by a factor of 4–40 times, depending on the task. Expand
Towards Motor Skill Learning for Robotics
TLDR
This paper proposes to break the generic skill learning problem into parts that the authors can understand well from a robotics point of view, and designs appropriate learning approaches for these basic components, which will serve as the ingredients of a general approach to motor skill learning. Expand
Socially guided intrinsic motivation for robot learning of motor skills
TLDR
It is illustrated that SGIM-D efficiently combines the advantages of social learning and intrinsic motivation and benefits from human demonstration properties to learn how to produce varied outcomes in the environment, while developing more precise control policies in large spaces. Expand
Reinforcement learning of motor skills in high dimensions: A path integral approach
TLDR
This paper derives a novel approach to RL for parameterized control policies based on the framework of stochastic optimal control with path integrals, and believes that this new algorithm, Policy Improvement with Path Integrals (PI2), offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL in robotics. Expand
Robot Skill Learning
TLDR
The generic skill learning problem is proposed to be divided into parts that can be well-understood from a robotics point of view, and appropriate learning approaches for these basic components will serve as the ingredients of a general approach to robot skill learning. Expand
Learning motor skills: from algorithms to robot experiments
  • J. Kober
  • Computer Science
  • it Inf. Technol.
  • 2012
TLDR
It is shown how motor primitives can be employed to learn motor skills on three different levels, which contributes to the state of the art in reinforcement learning applied to robotics both in terms of novel algorithms and applications. Expand
Learning Torque-Driven Manipulation Primitives with a Multilayer Neural Network
Autonomous learning of robotic motion skills promises to allow robots to acquire large motion repertoires for interacting with the real world with minimal human intervention. However, most currentExpand
Deep Reinforcement Learning for Robotic Manipulation
TLDR
It is demonstrated that a recent deep reinforcement learning algorithm based on offpolicy training of deep Q-functions can scale to complex 3D manipulation tasks and can learn deep neural network policies efficiently enough to train on real physical robots. Expand
From dynamic movement primitives to associative skill memories
TLDR
This paper reviews how a particular computational approach to movementPrimitives, called dynamic movement primitives, can contribute to learning motor skills, and addresses imitation learning, generalization, trial-and-error learning by reinforcement learning, movement recognition, and control based on movement Primitives. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 92 REFERENCES
Machine Learning of Motor Skills for Robotics, Jan Peters
    Autonomous robots that can assist humans in situations of daily life have been a long standing vision of robotics, artificial intelligence, and cognitive sciences. A first step towards this goal isExpand
    Reinforcement Learning for Humanoid Robotics
    TLDR
    This paper discusses different approaches of reinforcement learning in terms of their applicability in humanoid robotics, and demonstrates that ‘vanilla’ policy gradient methods can be significantly improved using the natural policy gradient instead of the regular policy gradient. Expand
    Learning Attractor Landscapes for Learning Motor Primitives
    TLDR
    By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be generated without losing the stability properties of the canonical system. Expand
    Policy Gradient Methods for Robotics
    • Jan Peters, S. Schaal
    • Engineering, Computer Science
    • 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems
    • 2006
    TLDR
    An overview on learning with policy gradient methods for robotics with a strong focus on recent advances in the field is given and how the most recently developed methods can significantly improve learning performance is shown. Expand
    Learning Movement Primitives
    TLDR
    A novel reinforcement learning technique based on natural stochastic policy gradients allows a general approach of improving DMPs by trial and error learning with respect to almost arbitrary optimization criteria, and demonstrates the different ingredients of the DMP approach in various examples. Expand
    Reinforcement learning for imitating constrained reaching movements
    TLDR
    A system to teach the robot constrained reaching tasks is described based on a dynamic system generator modulated by a learned speed trajectory combined with a reinforcement learning module to allow the robot to adapt the trajectory when facing a new situation, e.g., in the presence of obstacles. Expand
    Efficient Gradient Estimation for Motor Control Learning
    TLDR
    Two techniques for reducing gradient estimation errors in the presence of observable input noise applied to the control signal are presented and significantly improve the response function gradient estimate and, consequently, the learning curve, over existing methods. Expand
    Reinforcement Learning for CPG-Driven Biped Robot
    TLDR
    This study proposes a learning scheme for a CPG controller called a C PG-actor-critic model, whose learning algorithm is based on a policy gradient method, and applies this method to autonomous acquisition of biped locomotion by a biped robot simulator. Expand
    Acquiring robot skills via reinforcement learning
    TLDR
    A stochastic real-valued (SRV) reinforcement learning algorithm is described and used for learning control and the authors show how it can be used with nonlinear multilayer ANNs. Expand
    Neuronlike adaptive elements that can solve difficult learning control problems
    TLDR
    It is shown how a system consisting of two neuronlike adaptive elements can solve a difficult learning control problem and the relation of this work to classical and instrumental conditioning in animal learning studies and its possible implications for research in the neurosciences. Expand
    ...
    1
    2
    3
    4
    5
    ...