Corpus ID: 143422950

Billion-scale semi-supervised learning for image classification

@article{Yalniz2019BillionscaleSL,
  title={Billion-scale semi-supervised learning for image classification},
  author={I. Z. Yalniz and H. J{\'e}gou and Kan Chen and Manohar Paluri and D. Mahajan},
  journal={ArXiv},
  year={2019},
  volume={abs/1905.00546}
}
This paper presents a study of semi-supervised learning with large convolutional networks. [...] Key Result For instance, by leveraging one billion unlabelled images, our learned vanilla ResNet-50 achieves 81.2% top-1 accuracy on the ImageNet benchmark.Expand

Paper Mentions

A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification
TLDR
This work suggests that semi-supervised learning with experts on realistic datasets may require different strategies than those currently prevalent in the literature, and suggests that standard fine-tuning followed by distillation-based self-training is the most robust. Expand
STAR: Noisy Semi-Supervised Transfer Learning for Visual Classification
Semi-supervised learning (SSL) has proven to be effective at leveraging large-scale unlabeled data to mitigate the dependency on labeled data in order to learn better models for visual recognitionExpand
Improving Semantic Segmentation via Self-Training
TLDR
This paper demonstrates the effectiveness of self-training on a challenging cross-domain generalization task, outperforming conventional finetuning method by a large margin and proposes a fast training schedule to accelerate the training of segmentation models by up to 2x without performance degradation. Expand
Progressive Representative Labeling for Deep Semi-Supervised Learning
Deep semi-supervised learning (SSL) has experienced significant attention in recent years, to leverage a huge amount of unlabeled data to improve the performance of deep learning with limited labeledExpand
Learning from Few Samples: A Survey
TLDR
This paper provides a taxonomy for the techniques and categorize them as data-augmentation, embedding, optimization and semantics based learning for few-shot, one-shot and zero-shot settings, and describes the seminal work done in each category and discusses their approach towards solving the predicament of learning from few samples. Expand
VideoSSL: Semi-Supervised Learning for Video Classification
TLDR
It is shown that, under the supervision of these guiding signals from unlabeled examples, a video classification CNN can achieve impressive performances utilizing a small fraction of annotated examples on three publicly available datasets: UCF101, HMDB51, and Kinetics. Expand
Local Label Propagation for Large-Scale Semi-Supervised Learning
TLDR
This paper introduces a novel approach to scalable semi-supervised learning, called Local Label Propagation (LLP), which first embeds datapoints, labelled and otherwise, in a common latent space using a deep neural network, and propagates pseudolabels from known to unknown datapointeds in a manner that depends on the local geometry of the embedding. Expand
Semi-supervised Learning with a Teacher-student Network for Generalized Attribute Prediction
TLDR
This paper proposes a multi-teacher-single-student (MTSS) approach inspired by the multi-task learning and the distillation of semi-supervised learning that achieves competitive performance on various benchmarks for fashion attribute prediction, but also improves robustness and cross-domain adaptability for unseen domains. Expand
Generalized Many-Way Few-Shot Video Classification
TLDR
A simple 3D CNN baseline is developed, surpassing existing methods by a large margin and proposed to leverage weakly-labeled videos from a large dataset using tag retrieval followed by selecting the best clips with visual similarities, yielding further improvement. Expand
Out-distribution aware Self-training in an Open World Setting
TLDR
While normal self-training deteriorates prediction performance, the iterative scheme improves using up to 15 times the amount of originally labeled data and the classifiers are by design out-distribution aware and can thus distinguish task-related inputs from unrelated ones. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 44 REFERENCES
Low-Shot Learning with Large-Scale Diffusion
TLDR
This paper considers the problem of inferring image labels from images when only a few annotated examples are available at training time and considers a semi-supervised setting based on a large collection of images to support label propagation. Expand
Exploring the Limits of Weakly Supervised Pretraining
TLDR
This paper presents a unique study of transfer learning with large convolutional networks trained to predict hashtags on billions of social media images and shows improvements on several image classification and object detection tasks, and reports the highest ImageNet-1k single-crop, top-1 accuracy to date. Expand
Towards good practice in large-scale learning for image classification
TLDR
It is shown that for one-vs-rest, learning through cross-validation the optimal degree of imbalance between the positive and the negative samples can have a significant impact and early stopping can be used as an effective regularization strategy when training with stochastic gradient algorithms. Expand
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
TLDR
This work designs a method to reuse layers trained on the ImageNet dataset to compute mid-level image representation for images in the PASCAL VOC dataset, and shows that despite differences in image statistics and tasks in the two datasets, the transferred representation leads to significantly improved results for object and action classification. Expand
Label Refinery: Improving ImageNet Classification through Label Progression
TLDR
The effects of various properties of labels are studied, an iterative procedure that updates the ground truth labels after examining the entire dataset is introduced, and significant gain is shown using refined labels across a wide range of models. Expand
Learning feed-forward one-shot learners
TLDR
This paper constructs the learner as a second deep network, called a learnet, which predicts the parameters of a pupil network from a single exemplar, and obtains an efficient feed-forward one-shot learner, trained end-to-end by minimizing a one- shot classification objective in a learning to learn formulation. Expand
Rethinking ImageNet Pre-Training
TLDR
Experiments show that ImageNet pre-training speeds up convergence early in training, but does not necessarily provide regularization or improve final target task accuracy, and these discoveries will encourage people to rethink the current de facto paradigm of `pre-training and fine-tuning' in computer vision. Expand
Transfer Learning in a Transductive Setting
TLDR
This work extends transfer learning with semi-supervised learning to exploit unlabeled instances of (novel) categories with no or only a few labeled instances to exploit the manifold structure of novel classes. Expand
Transformation Pursuit for Image Classification
TLDR
This work proposes a principled algorithm -- Image Transformation Pursuit (ITP) -- for the automatic selection of a compact set of transformations, by selecting at each iteration the one that yields the highest accuracy gain. Expand
Low-Shot Visual Recognition by Shrinking and Hallucinating Features
TLDR
This work presents a low-shot learning benchmark on complex images that mimics challenges faced by recognition systems in the wild, and proposes representation regularization techniques and techniques to hallucinate additional training examples for data-starved classes. Expand
...
1
2
3
4
5
...