Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation

@article{Liang2021SearchBasedTP,
  title={Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation},
  author={Jacky Liang and Mohit Sharma and Alex Licari LaGrassa and Shivam Vats and Saumya Saxena and Oliver Kroemer},
  journal={2022 International Conference on Robotics and Automation (ICRA)},
  year={2021},
  pages={6351-6357}
}
Robots deployed in many real-world settings need to be able to acquire new skills and solve new tasks over time. Prior works on planning with skills often make assumptions on the structure of skills and tasks, such as subgoal skills, shared skill implementations, or task-specific plan skeletons, which limit adaptation to new skills and tasks. By contrast, we propose doing task planning by jointly searching in the space of parameterized skills using high-level skill effect models learned in… 

Figures and Tables from this paper

ULATIONS WITH TOOLS

This work proposes a novel framework, named DiffSkill, that uses a differentiable physics simulator for skill abstraction to solve long-horizon deformable object manipulation tasks from sensory observations and shows the advantages of the method in a new set of sequential deformable objects manipulation tasks compared to previous reinforcement learning algorithms and compared to the trajectory optimizer.

Generalizable Task Planning Through Representation Pretraining

This letter proposes a learning-to-plan method that can generalize to new object instances by leveraging object-level representations extracted from a synthetic scene understanding dataset and shows that the model achieves measurably better success rate than state-of-the-art end- to-end approaches.

Simulation-based Learning of the Peg-in-Hole Process Using Robot-Skills

A solution to learn parameters for contact-rich, force-controlled assembly tasks from a simulation using hardware-independent robot skills and it is shown that successful learning and real-world execution are possible even under process deviation and tolerances utilizing the designed learning system.

DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools

This work proposes a novel framework, named DiffSkill, that uses a differentiable physics simulator for skill abstraction to solve long-horizon deformable object manipulation tasks from sensory observations and shows the advantages of the method in a new set of sequential deformable objects manipulation tasks compared to previous reinforcement learning algorithms and compared to the trajectory optimizer.

Robust Planning for Multi-stage Forceful Manipulation

This work augments an existing task and motion planner with constraints that explicitly consider torque and frictional limits, captured through the proposed forceful kinematic chain constraint, and demonstrates how the system selects from among a combinatorial set of strategies.

Planning with Learned Model Preconditions for Water Manipulation

This work addresses the problem of modeling deformable object dynamics by learning where a set of given high-level dynamics models are accurate: a model precondition, which is then used to model trajectories using states and closed-loop actions where the dynamics model are accurate.

Learning Preconditions of Hybrid Force-Velocity Controllers for Contact-Rich Manipulation

This work first relax HFVCs’ need for precise models and feedback with their HFVC synthesis framework, then learns a point-cloud-based precondition function to classify where HFVC executions will still be successful despite modeling inaccuracies, and uses the learned preconditions in a search-based task planner to complete contact-rich manipulation tasks in a shelf domain.

Library of behaviors and tools for robot manipulation

A visually-grounded library of behaviors approach for learning to manipulate diverse objects across varying initial and goal configurations and camera placements and an end-to-end learning framework that jointly learns to choose different tools and deploy tool-conditioned policies directly on a real robot platform.

Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers

A novel graph neural network framework for multi-object manipulation to predict how inter-object relations change given robot actions, which enables multi-step planning to reach target goal relations and shows the model trained purely in simulation transfers well to the real world.

Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulation

This paper proposes PlAnning with Spatial and Temporal Abstraction (PASTA), which incorporates both spatial abstraction (reasoning about objects and their relations to each other) and temporal abstraction ( Reasoning over skills instead of low-level actions).

References

SHOWING 1-10 OF 57 REFERENCES

Learning compositional models of robot skills for task and motion planning

This work uses Gaussian process methods for learning the constraints on skill effectiveness from small numbers of expensive-to-collect training examples and develops efficient adaptive sampling methods for generating a comprehensive and diverse sequence of continuous candidate control parameter values during planning.

Skill Transfer via Partially Amortized Hierarchical Planning

This paper leverages the idea of partial amortization for fast adaptation at test time in single tasks as well as in transfer from one task to another, as compared to competitive baselines.

Learning Symbolic Representations for Planning with Parameterized Skills

This work shows how to construct a representation suitable for planning with parametrized motor skills, and specifies conditions which are sufficient to separate the selection of motor skills from the parametrization of those skills.

Reset-Free Lifelong Learning with Skill-Space Planning

Lifelong Skill Planning (LiSP), an algorithmic framework for non-episodic lifelong RL based on planning in an abstract space of higher-order skills, successfully enables long-horizon planning and learns agents that can avoid catastrophic failures even in challenging non-stationary and non-EPisodic environments derived from gridworld and MuJoCo benchmarks.

PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning

This work provides domain-independent algorithms that reduce PDDLStream problems to a sequence of finite PDDL problems and introduces an algorithm that dynamically balances exploring new candidate plans and exploiting existing ones to solve tightly-constrained problems.

Deep Affordance Foresight: Planning Through What Can Be Done in the Future

A new affordance representation is introduced that enables the robot to reason about the longterm effects of actions through modeling what actions are afforded in the future and develops a learning-to-plan method, Deep Affordance Foresight (DAF), that learns partial environment models of affordances of parameterized motor skills through trial-and-error.

Planning with Goal-Conditioned Policies

This work shows that goal-conditioned policies learned with RL can be incorporated into planning, such that a planner can focus on which states to reach, rather than how those states are reached, and proposes using a latent variable model to compactly represent the set of valid states.

Lifelong robot learning

Policy search for motor primitives in robotics

A novel EM-inspired algorithm for policy learning that is particularly well-suited for dynamical system motor primitives is introduced and applied in the context of motor learning and can learn a complex Ball-in-a-Cup task on a real Barrett WAM™ robot arm.

Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion

This letter presents a fully-learned hierarchical framework, that is capable of jointly learning the low-level controller and the high-level latent action space, and shows that this framework outperforms baselines on multiple tasks and two simulations.
...