Corpus ID: 237500490

Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation

  title={Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation},
  author={Todor Davchev and Kevin Sebastian Luck and Michael Burke and Franziska Meier and Stefan Schaal and Subramanian Ramamoorthy},
Manipulation skills involving contact and friction are inherent to many robotics tasks. Using the class of motor primitives for peg-in-hole like insertions, we study how robots can learn such skills. Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC), but can struggle in the context of insertion. Policy adaptation strategies such as residual learning can help improve the overall performance of policies in the context of contact-rich… Expand

Figures and Tables from this paper


Policy search for motor primitives in robotics
A novel EM-inspired algorithm for policy learning that is particularly well-suited for dynamical system motor primitives is introduced and applied in the context of motor learning and can learn a complex Ball-in-a-Cup task on a real Barrett WAM™ robot arm. Expand
Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
This paper considers a variety of difficult industrial insertion tasks with visual inputs and different natural reward specifications, namely sparse rewards and goal images and shows that methods that combine RL with prior information can solve these tasks from a reasonable amount of real-world interaction. Expand
Reinforcement learning of motor skills with policy gradients
This paper examines learning of complex motor skills with human-like limbs, and combines the idea of modular motor control by means of motor primitives as a suitable way to generate parameterized control policies for reinforcement learning with the theory of stochastic policy gradient learning. Expand
Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
A general and model-free approach for Reinforcement Learning on real robotics with sparse rewards built upon the Deep Deterministic Policy Gradient algorithm to use demonstrations that out-performs DDPG, and does not require engineered rewards. Expand
Learning by Demonstration
  • S. Schaal
  • Computer Science
  • Encyclopedia of Machine Learning and Data Mining
  • 2017
In an implementation of pole balancing on a complex anthropomorphic robot arm, it is demonstrated that, when facing the complexities of real signal processing, model-based reinforcement learning offers the most robustness for LQR problems. Expand
Reinforcement Learning With Sequences of Motion Primitives for Robust Manipulation
Applying simultaneous shape and goal learning to sequences of motion primitives leads to the novel algorithm PI2 Seq, which is used to address a fundamental challenge in manipulation: improving the robustness of everyday pick-and-place tasks. Expand
Motion planning and reactive control on learnt skill manifolds †
This work addresses the problem of encoding and executing skills, i.e. motion tasks involving a combination of specifications regarding constraints and variability, by encoding trajectories in a skill manifold which is learnt from data and generalizes in the sense that all trajectories on the manifold satisfy the constraints and allowable variability in the demonstrated samples. Expand
Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks
A general data-driven framework for learning a feedback model from demonstrations using a variant of a radial basis function network structure –with movement phases as kernel centers– which can generally be applied to represent any feedback models for movement primitives. Expand
Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning
This work uses uncertainty estimates to divide the space in regions where the given model-based policy is reliable, and regions where it may have flaws or not be well defined, and shows that a locally learned-policy can be used directly with raw sensory inputs. Expand
Residual Reinforcement Learning for Robot Control
This paper studies how to solve difficult control problems in the real world by decomposing them into a part that is solved efficiently by conventional feedback control methods, and the residual which is solved with RL. Expand