Visually Grounded Task and Motion Planning for Mobile Manipulation

@inproceedings{Zhang2022VisuallyGT,
  title={Visually Grounded Task and Motion Planning for Mobile Manipulation},
  author={Xiaohan Zhang and Yifeng Zhu and Yan Ding and Yuke Zhu and Peter Stone and Shiqi Zhang},
  booktitle={ICRA},
  year={2022}
}
Task and motion planning (TAMP) algorithms aim to help robots achieve task-level goals, while maintaining motion-level feasibility. This paper focuses on TAMP domains that involve robot behaviors that take extended periods of time (e.g., long-distance navigation). In this paper, we develop a visual grounding approach to help robots probabilistically evaluate action feasibility, and introduce a TAMP algorithm, called GROP, that optimizes both feasibility and efficiency. We have collected a… 

Figures and Tables from this paper

Learning to Ground Objects for Robot Task and Motion Planning
TLDR
This letter defines a new object-centric TAMP problem, where the TAMP robot does not know object properties, and introduces Task-Motion Object-Centric planning (TMOC), a grounded TAMP algorithm that learns to ground objects and their physical properties with a physics engine.

References

SHOWING 1-10 OF 44 REFERENCES
Learning to Ground Objects for Robot Task and Motion Planning
TLDR
This letter defines a new object-centric TAMP problem, where the TAMP robot does not know object properties, and introduces Task-Motion Object-Centric planning (TMOC), a grounded TAMP algorithm that learns to ground objects and their physical properties with a physics engine.
Combined task and motion planning through an extensible planner-independent interface layer
TLDR
This work proposes a new approach that uses off-the-shelf task planners and motion planners and makes no assumptions about their implementation and uses a novel representational abstraction that requires only that failures in computing a motion plan for a high-level action be identifiable and expressible in the form of logical predicates at the task level.
Object-Centric Task and Motion Planning in Dynamic Environments
TLDR
This work applies the TAMP framework to a torque-controlled robot in a pick and place setting and demonstrates its ability to adapt to changing environments, inaccurate perception, and imprecise control, both in simulation and the real world.
Learning Feasibility for Task and Motion Planning in Tabletop Environments
TLDR
The key idea is to learn a classifier for feasible motions and to use this classifier as a heuristic to order the search for a task–motion plan, which reduces the total number of motion planning attempts.
An incremental constraint-based framework for task and motion planning
We present a new constraint-based framework for task and motion planning (TMP). Our approach is extensible, probabilistically complete, and offers improved performance and generality compared with a
Guided search for task and motion plans using learned heuristics
TLDR
This work presents an algorithm that searches the space of possible task and motion plans and uses statistical machine learning to guide the search process and shows significant improvements in performance over prior work.
Task-Motion Planning with Reinforcement Learning for Adaptable Mobile Service Robots
TLDR
A novel framework, TMP-RL, which is an integration of TMP and reinforcement learning (RL), to solve the problem of robust TMP in dynamic and uncertain domains is proposed and evaluated in a mobile service robot domain, showing significantly improved adaptability to unseen domain dynamics over T MP and task planning (TP)-RL methods.
Integrated Task and Motion Planning
TLDR
This paper defines a class of TAMP problems and survey algorithms for solving them, characterizing the solution methods in terms of their strategies for solving the continuous-space subproblems and their techniques for integrating the discrete and continuous components of the search.
Efficiently combining task and motion planning using geometric constraints
TLDR
A constraint-based approach is proposed to address a class of problems encountered in combined task and motion planning, which is called kinematically constrained problems, where a significant computational effort is spent on geometric backtrack search, which impairs search at the task level.
...
...