Self-Supervised Correspondence in Visuomotor Policy Learning

  title={Self-Supervised Correspondence in Visuomotor Policy Learning},
  author={P. Florence and Lucas Manuelli and Russ Tedrake},
  journal={IEEE Robotics and Automation Letters},
  • P. Florence, Lucas Manuelli, Russ Tedrake
  • Published 2020
  • Computer Science
  • IEEE Robotics and Automation Letters
  • In this letter, we explore using self-supervised correspondence for improving the generalization performance and sample efficiency of visuomotor policy learning. Prior work has primarily used approaches such as autoencoding, pose-based losses, and end-to-end policy optimization in order to train the visual portion of visuomotor policies. We instead propose an approach using self-supervised dense visual correspondence training and show that this enables visuomotor policy learning with… CONTINUE READING
    15 Citations

    Figures, Tables, and Topics from this paper.

    Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning
    • 1
    • PDF
    A Geometric Perspective on Visual Imitation Learning
    Generalization Guarantees for Multi-Modal Imitation Learning
    Transporter Networks: Rearranging the Visual World for Robotic Manipulation
    • 2
    • PDF
    MELD: Meta-Reinforcement Learning from Images via Latent State Models
    S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistency


    Reinforcement and Imitation Learning for Diverse Visuomotor Skills
    • 137
    • PDF
    End-to-End Training of Deep Visuomotor Policies
    • 1,875
    • Highly Influential
    • PDF
    GPLAC: Generalizing Vision-Based Robotic Skills Using Weakly Labeled Images
    • 18
    • PDF
    Deep spatial autoencoders for visuomotor learning
    • 286
    • Highly Influential
    • PDF
    Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
    • 98
    • PDF
    Unsupervised Visuomotor Control through Distributional Planning Networks
    • 25
    • PDF
    Stable reinforcement learning with autoencoders for tactile and visual data
    • 81
    • PDF
    One-Shot Visual Imitation Learning via Meta-Learning
    • 229
    • PDF
    Deep predictive policy training using reinforcement learning
    • 73
    • PDF
    Graph-Structured Visual Imitation
    • 7
    • PDF