An Algorithmic Perspective on Imitation Learning

@article{Osa2018AnAP,
  title={An Algorithmic Perspective on Imitation Learning},
  author={Takayuki Osa and J. Pajarinen and G. Neumann and J. Bagnell and P. Abbeel and Jan Peters},
  journal={Found. Trends Robotics},
  year={2018},
  volume={7},
  pages={1-179}
}
As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It… Expand
Back to Reality for Imitation Learning
  • 2021
Imitation learning, and robot learning in general, emerged due to break1 throughs in machine learning, rather than breakthroughs in robotics. As such, 2 evaluation metrics for robot learning areExpand
Metric-Based Imitation Learning Between Two Dissimilar Anthropomorphic Robotic Arms
TLDR
This work addresses the correspondence problem in imitation learning by introducing a distance measure and finds that the measure is well suited for describing the similarity between embodiments and for learning imitation policies by distance minimization. Expand
ROBUST IMITATION VIA DECISION-TIME PLANNING
  • 2020
The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal. A popular class of approach infers the (unknown) reward function viaExpand
Task-Oriented Deep Reinforcement Learning for Robotic Skill Acquisition and Control
TLDR
An efficient model-free off-policy actor–critic algorithm for robotic skill acquisition and continuous control is presented, by fusing the task reward with a task-oriented guiding reward, which is formulated by leveraging few and imperfect expert demonstrations. Expand
Imitation Learning: Progress, Taxonomies and Opportunities
TLDR
This survey provides a systematic review on imitation learning, introducing the background knowledge from development history and preliminaries, followed by presenting different taxonomies within Imitation Learning and key milestones of the field. Expand
Survey of imitation learning for robotic manipulation
TLDR
The survey of imitation learning of robotic manipulation involves three aspects that are demonstration, representation and learning algorithms and highlights areas of future research potential. Expand
Modeling Task Uncertainty for Safe Meta-Imitation Learning
TLDR
A novel framework for estimating the task uncertainty through probabilistic inference in the task-embedding space, called PETNet is proposed, which can achieve the same or higher level of performance (success rate of novel tasks at meta-test time) as previous methods. Expand
Benchmarking Reinforcement Learning Algorithms on Tetherball Games
Robotic applications in the real world are faced with the stochasticity and uncertainty of the environment, which renders the approaches of having hand-tuned controllers impractical. ReinforcementExpand
Recent Advances in Imitation Learning from Observation
TLDR
A literature review of methods developed for imitation from observation (IfO) is provided, and some open research problems and potential future work are pointed out. Expand
Regression via Kirszbraun Extension with Applications to Imitation Learning
TLDR
This work presents a general, model free and non-parametric imitation learning algorithm based on regression between two Hilbert spaces based on Kirszbraun's extension theorem, and applies it to a static posture imitation task between two robotic manipulators with different embodiments, and reports promising results. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 237 REFERENCES
Learning to search: Functional gradient techniques for imitation learning
TLDR
The work presented extends the Maximum Margin Planning (MMP) framework to admit learning of more powerful, non-linear cost functions, and demonstrates practical real-world performance with three applied case-studies including legged locomotion, grasp planning, and autonomous outdoor unstructured navigation. Expand
Efficient Reductions for Imitation Learning
TLDR
This work proposes two alternative algorithms for imitation learning where training occurs over several episodes of interaction and shows that this leads to stronger performance guarantees and improved performance on two challenging problems: training a learner to play a 3D racing game and Mario Bros. Expand
An Invitation to Imitation
TLDR
This work represents a summary from a very personal perspective of research on computationally effective methods for learning to imitate behavior to serve two audiences: to engage machine learning experts in the challenges of imitation learning and the interesting theoretical and practical distinctions with more familiar frameworks like statistical supervised learning theory. Expand
Reinforcement learning in robotics: A survey
TLDR
This article attempts to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots by highlighting both key challenges in robot reinforcement learning as well as notable successes. Expand
Third-Person Imitation Learning
TLDR
The methods primary insight is that recent advances from domain confusion can be utilized to yield domain agnostic features which are crucial during the training process. Expand
Incremental imitation learning of context-dependent motor skills
TLDR
This paper addresses challenges by presenting a method to incrementally teach context-dependent motor skills to robots and evaluated the method with experiments in which an elastically actuated robot arm with four degrees of freedom learns how to reach a ball at different positions. Expand
Learning Nonparametric Models for Probabilistic Imitation
TLDR
A new probabilistic method for inferring imitative actions that takes into account both the observations of the teacher as well as the imitator's dynamics, which generalizes to systems with very different dynamics. Expand
Gaussian Processes for Data-Efficient Learning in Robotics and Control
TLDR
This paper learns a probabilistic, non-parametric Gaussian process transition model of the system and applies it to autonomous learning in real robot and control tasks, achieving an unprecedented speed of learning. Expand
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
TLDR
This work proposes an imitation learning method based on video prediction with context translation and deep reinforcement learning that enables a variety of interesting applications, including learning robotic skills that involve tool use simply by observing videos of human tool use. Expand
Model-Free Imitation Learning with Policy Optimization
TLDR
Under the apprenticeship learning formalism, this work develops alternative model-free algorithms for finding a parameterized stochastic policy that performs at least as well as an expert policy on an unknown cost function, based on sample trajectories from the expert. Expand
...
1
2
3
4
5
...