Corpus ID: 231632575

Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos

@article{Xiong2021LearningBW,
  title={Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos},
  author={Haoyu Xiong and Quanzhou Li and Yun-Chun Chen and Homanga Bharadhwaj and Samarth Sinha and Animesh Garg},
  journal={ArXiv},
  year={2021},
  volume={abs/2101.07241}
}
We present an approach for physical imitation from human videos for robot manipulation tasks. The key idea of our method lies in explicitly exploiting the kinematics and motion information embedded in the video to learn structured representations that endow the robot with the ability to imagine how to perform manipulation tasks in its own context. To achieve this, we design a perception module that learns to translate human videos to the robot domain followed by unsupervised keypoint detection… Expand
1 Citations

Figures and Tables from this paper

Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos
  • PDF

References

SHOWING 1-10 OF 48 REFERENCES
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
  • 173
  • PDF
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
  • 160
  • Highly Influential
  • PDF
Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller
  • 16
  • PDF
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
  • 211
  • PDF
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos
  • 19
  • Highly Influential
  • PDF
SFV: Reinforcement Learning of Physical Skills from Videos
  • 91
  • PDF
Zero-Shot Visual Imitation
  • 155
  • PDF
Time-Contrastive Networks: Self-Supervised Learning from Video
  • 257
  • PDF
...
1
2
3
4
5
...