Extracting bimanual synergies with reinforcement learning

  title={Extracting bimanual synergies with reinforcement learning},
  author={Kevin Sebastian Luck and Heni Ben Amor},
  journal={2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  • K. Luck, H. B. Amor
  • Published 1 September 2017
  • Computer Science
  • 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Motor synergies are an important concept in human motor control. Through the co-activation of multiple muscles, complex motion involving many degrees-of-freedom can be generated. However, leveraging this concept in robotics typically entails using human data that may be incompatible for the kinematics of the robot. In this paper, our goal is to enable a robot to identify synergies for low-dimensional control using trial-and-error only. We discuss how synergies can be learned through latent… 

Discovering Synergies for Robot Manipulation with Multi-Task Reinforcement Learning

This paper presents a framework that simultaneously discovers both a synergy space and a multi-task policy that operates on this low-dimensional action space to accomplish diverse manipulation tasks and demonstrates that the end-to-end method is able to perform multiple tasks using few synergies, and outperforms sequential methods that apply dimensionality reduction to independently collected data.

A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies

A framework for teaching bimanual manipulation tasks by imitation combines insights from admittance control and machine learning to extract control policies that can recover from and adapt to a variety of disturbances in time and space, while also leveraging physical contact with the environment.

Exploiting Symmetries in Reinforcement Learning of Bimanual Robotic Tasks

A symmetrization method for ProMPs is presented and used to represent two movements, employing a single ProMP for the first arm and a symmetry surface that maps that ProMP to the second arm, which is adopted in reinforcement learning of bimanual tasks using relative entropy policy search algorithm.

Improved Exploration through Latent Trajectory Optimization in Deep Deterministic Policy Gradient

This work evaluates the use of model-based trajectory optimization methods used for exploration in Deep Deterministic Policy Gradient when trained on a latent image embedding, leading to a symbiotic relationship between the deep reinforcement learning algorithm and the latent trajectory optimizer.



Extracting motor synergies from random movements for low-dimensional task-space control of musculoskeletal robots

This paper proposes to extract motor synergies from a subset of randomly generated reaching-like movement data, and presents a kernel-based regression formulation to estimate the forward and the inverse dynamics, and a sliding controller in order to cope with estimation error.

Kinesthetic Bootstrapping: Teaching Motor Skills to Humanoid Robots through Physical Interaction

A new programming-by-demonstration method called Kinesthetic Bootstrapping for teaching motor skills to humanoid robots by means of intuitive physical interactions, which has been successfully applied to the learning of various complex motor skills such as walking and standing up.

Policy search for motor primitives in robotics

A novel EM-inspired algorithm for policy learning that is particularly well-suited for dynamical system motor primitives is introduced and applied in the context of motor learning and can learn a complex Ball-in-a-Cup task on a real Barrett WAM™ robot arm.

Latent space policy search for robotics

This paper presents a novel policy search method that performs efficient reinforcement learning by uncovering the low-dimensional latent space of actuator redundancies by performing dimensionality reduction as a preprocessing step but naturally combines it with policy search.

Hand Posture Subspaces for Dexterous Robotic Grasping

An on-line grasp planner that allows a human operator to perform dexterous grasping tasks using an artificial hand in a hand posture subspace of highly reduced dimensionality is presented.

Subject-specific muscle synergies in human balance control are consistent across different biomechanical contexts.

It is shown that trial-by-trial variations in muscle activation for multidirectional balance control in humans were constrained by a small set of muscle synergies, and Muscle synergies represent consistent motor modules that map intention to action, regardless of the biomechanical context of the task.

Sparse Latent Space Policy Search

A reinforcement learning method for sample-efficient policy search that exploits correlations between control variables, particularly frequent in motor skill learning tasks, and outperforms state-of-the-art policy search methods.

Goal directed multi-finger manipulation: Control policies and analysis

Identification of muscle synergies associated with gait transition in humans

The results suggest that the CNS low-dimensionally regulate the activation profiles of the specific synergies based on afferent information (spontaneous gait Transition) or by changing only the descending neural input to the muscle synergies (voluntary gait transition) to achieve a gait transitions.

Postural Hand Synergies for Tool Use

The results suggest that the control of hand posture involves a few postural synergies, regulating the general shape of the hand, coupled with a finer control mechanism providing for small, subtle adjustments.