# Sparse Latent Space Policy Search

@inproceedings{Luck2016SparseLS, title={Sparse Latent Space Policy Search}, author={Kevin Sebastian Luck and Joni Pajarinen and Erik Berger and Ville Kyrki and Heni Ben Amor}, booktitle={AAAI}, year={2016} }

Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sample-efficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method…

## 14 Citations

Information Maximizing Exploration with a Latent Dynamics Model

- Computer ScienceArXiv
- 2018

This work presents an approach that uses a model to derive reward bonuses as a means of intrinsic motivation to improve model-free reinforcement learning and is both theoretically grounded and computationally advantageous, permitting the efficient use of Bayesian information-theoretic methods in high-dimensional state spaces.

Multimodal Policy Search using Overlapping Mixtures of Sparse Gaussian Process Prior

- Computer Science2019 International Conference on Robotics and Automation (ICRA)
- 2019

A novel policy search reinforcement learning algorithm that can deal with multimodality in control policies based on Gaussian processes by placing the OMSGPs as the prior of the multimodal control policy.

Latent Space Reinforcement Learning for Steering Angle Prediction

- Computer ScienceArXiv
- 2019

This work addresses the problem of learning driving policies for an autonomous agent in a high-fidelity simulator with a modular deep reinforcement learning approach to predict the steering angle of the car from raw images.

Motor Synergy Development in High-Performing Deep Reinforcement Learning Algorithms

- Computer ScienceIEEE Robotics and Automation Letters
- 2020

This is the first attempt to quantify the synergy development in detail and evaluate its emergence process during deep learning motor control tasks and it is demonstrated that there is a correlation between the synergy-related metrics and the performance and energy efficiency of a trained agent.

Extracting bimanual synergies with reinforcement learning

- Computer Science2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- 2017

It is discussed how synergies can be learned through latent space policy search and an extension of the algorithm for the re-use of previously learned synergies for exploration is introduced and introduced.

Sample-Efﬁcient Reinforcement Learning for Robot to Human Handover Tasks

- Computer Science
- 2017

This work used the Sparse Latent Space Policy Search algorithm and a linear-Gaussian trajectory approximator with the objective of learning optimized, understandable trajectories for object handovers between a robot and a human with very high sample-efﬁciency.

xtracting Bimanual Synergies with Reinforcement Learning

- Computer Science
- 2017

It is discussed how synergies can be learned through latent space policy search and an extension of the algorithm for the re-use of previously learned synergies for exploration is introduced and introduced.

Disturbance-Injected Robust Imitation Learning with Task Achievement

- Computer Science2022 International Conference on Robotics and Automation (ICRA)
- 2022

This paper proposes a novel imitation learning framework that combines both policy robustification and optimal demonstration learning, and forces policy learning and disturbance injection optimization to focus on mainly learning from high task achievement demonstrations, while utilizing low achievement ones to decrease the number of samples needed.

From the Lab to the Desert: Fast Prototyping and Learning of Robot Locomotion

- Computer ScienceRobotics: Science and Systems
- 2017

The findings of this study show that static policies developed in the laboratory do not translate to effective locomotion strategies in natural environments, and sample-efficient reinforcement learning can help to rapidly accommodate changes in the environment or the robot.

Bi-manual Learning for a Basketball Playing Robot

- Computer Science
- 2016

A dual armed robot has been built and taught to handle the ball and make the basket successfully thus demonstrating the capability of using both arms.

## References

SHOWING 1-10 OF 24 REFERENCES

Latent space policy search for robotics

- Computer Science2014 IEEE/RSJ International Conference on Intelligent Robots and Systems
- 2014

This paper presents a novel policy search method that performs efficient reinforcement learning by uncovering the low-dimensional latent space of actuator redundancies by performing dimensionality reduction as a preprocessing step but naturally combines it with policy search.

Using dimensionality reduction to exploit constraints in reinforcement learning

- Computer Science2010 IEEE/RSJ International Conference on Intelligent Robots and Systems
- 2010

This work presents an alternative way to incorporate prior knowledge from demonstrations of individual postures into learning, by extracting the inherent problem structure to find an efficient state representation in the learnt latent space.

Policy search for motor primitives in robotics

- Computer ScienceMachine Learning
- 2010

A novel EM-inspired algorithm for policy learning that is particularly well-suited for dynamical system motor primitives is introduced and applied in the context of motor learning and can learn a complex Ball-in-a-Cup task on a real Barrett WAM™ robot arm.

Variational Inference for Policy Search in changing situations

- Computer ScienceICML
- 2011

Variational Inference for Policy Search (VIP) has several interesting properties and meets the performance of state-of-the-art methods while being applicable to simultaneously learning in multiple situations.

Learning omnidirectional path following using dimensionality reduction

- Computer ScienceRobotics: Science and Systems
- 2007

A method is proposed that uses a (possibly inaccurate) simulator to identify a low-dimensional subspace of policies that spans the variations in model dynamics that can be learned on the real system using much less data than would be required to learn a policy in the original class.

Robot trajectory optimization using approximate inference

- Computer Science, MathematicsICML '09
- 2009

This work considers a probabilistic model for which the maximum likelihood (ML) trajectory coincides with the optimal trajectory and which reproduces the classical SOC solution and utilizes approximate inference methods that efficiently generalize to non-LQG systems.

Black-Box Policy Search with Probabilistic Programs

- Computer ScienceAISTATS
- 2016

This work relates classic policy gradient techniques to recently introduced black-box variational methods which generalize to probabilistic program inference and presents case studies in the Canadian traveler problem, Rock Sample, and a benchmark for optimal diagnosis inspired by Guess Who.

Group Factor Analysis

- Computer ScienceIEEE Transactions on Neural Networks and Learning Systems
- 2015

The resulting solution solves the group factor analysis (GFA) problem accurately, outperforming alternative FA-based solutions as well as more straightforward implementations of GFA.

Postural Hand Synergies for Tool Use

- Psychology, BiologyThe Journal of Neuroscience
- 1998

The results suggest that the control of hand posture involves a few postural synergies, regulating the general shape of the hand, coupled with a finer control mechanism providing for small, subtle adjustments.