How Useful Is Self-Supervised Pretraining for Visual Tasks?

@article{Newell2020HowUI,
  title={How Useful Is Self-Supervised Pretraining for Visual Tasks?},
  author={Alejandro Newell and Jun Deng},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={7343-7352}
}
  • Alejandro Newell, Jun Deng
  • Published 2020
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Recent advances have spurred incredible progress in self-supervised pretraining for vision. We investigate what factors may play a role in the utility of these pretraining methods for practitioners. To do this, we evaluate various self-supervised algorithms across a comprehensive array of synthetic datasets and downstream tasks. We prepare a suite of synthetic data that enables an endless supply of annotated images as well as full control over dataset difficulty. Our experiments offer insights… Expand
23 Citations
Transfer Learning or Self-supervised Learning? A Tale of Two Pretraining Paradigms
  • 3
  • PDF
Self-Supervised Pretraining Improves Self-Supervised Pretraining
  • PDF
Streaming Self-Training via Domain-Agnostic Unlabeled Images
  • PDF
Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
  • PDF
Aggregative Self-Supervised Feature Learning
  • PDF
Learning Visual Representations for Transfer Learning by Suppressing Texture
  • Highly Influenced
  • PDF
F EATURE S PACES FOR R EP-RESENTATION L EARNING
...
1
2
3
...

References

SHOWING 1-10 OF 55 REFERENCES
Scaling and Benchmarking Self-Supervised Visual Representation Learning
  • 132
  • PDF
Exploring the Limits of Weakly Supervised Pretraining
  • 531
  • PDF
The Visual Task Adaptation Benchmark
  • 26
  • PDF
Revisiting Self-Supervised Visual Representation Learning
  • 300
  • PDF
Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
  • 361
  • PDF
Probing the State of the Art: A Critical Look at Visual Representation Evaluation
  • 6
  • PDF
Unsupervised Learning by Predicting Noise
  • 153
  • PDF
Local Aggregation for Unsupervised Learning of Visual Embeddings
  • 147
  • PDF
Unsupervised Representation Learning by Predicting Image Rotations
  • 855
  • PDF
...
1
2
3
4
5
...