Self-Supervised Learning of Pretext-Invariant Representations

@article{Misra2020SelfSupervisedLO,
  title={Self-Supervised Learning of Pretext-Invariant Representations},
  author={I. Misra and L. V. D. Maaten},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={6706-6716}
}
  • I. Misra, L. V. D. Maaten
  • Published 2020
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • The goal of self-supervised learning from images is to construct image representations that are semantically meaningful via pretext tasks that do not require semantic annotations. Many pretext tasks lead to representations that are covariant with image transformations. We argue that, instead, semantic representations ought to be invariant under such transformations. Specifically, we develop Pretext-Invariant Representation Learning (PIRL, pronounced as `pearl') that learns invariant… CONTINUE READING

    Figures, Tables, and Topics from this paper.

    Paper Mentions

    Learning Representations by Predicting Bags of Visual Words
    5
    VirTex: Learning Visual Representations from Textual Annotations
    Delving into Inter-Image Invariance for Unsupervised Visual Representations
    Whitening for Self-Supervised Representation Learning
    Knowledge Distillation Meets Self-Supervision
    3
    Prototypical Contrastive Learning of Unsupervised Representations
    9
    Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
    1

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 96 REFERENCES
    Revisiting Self-Supervised Visual Representation Learning
    169
    Transitive Invariance for Self-Supervised Visual Representation Learning
    100
    Representation Learning by Learning to Count
    138
    Self-Supervised Representation Learning by Rotation Feature Decoupling
    37
    Boosting Self-Supervised Learning via Knowledge Transfer
    87
    Scaling and Benchmarking Self-Supervised Visual Representation Learning
    65
    Unsupervised Visual Representation Learning by Context Prediction
    813
    Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
    579