Barely-Supervised Learning: Semi-Supervised Learning with very few labeled images

@article{Lucas2022BarelySupervisedLS,
  title={Barely-Supervised Learning: Semi-Supervised Learning with very few labeled images},
  author={Thomas Lucas and Philippe Weinzaepfel and Gr{\'e}gory Rogez},
  journal={ArXiv},
  year={2022},
  volume={abs/2112.12004}
}
This paper tackles the problem of semi-supervised learning when the set of labeled samples is limited to a small number of images per class, typically less than 10, problem that we refer to as barely-supervised learning. We analyze in depth the behavior of a state-of-the-art semi-supervised method, FixMatch, which relies on a weakly-augmented version of an image to obtain supervision signal for a more strongly-augmented version. We show that it frequently fails in barely-supervised scenarios… 

Figures and Tables from this paper

ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization

. We present a novel semi-supervised learning framework that intelligently leverages the consistency regularization between the model’s predictions from two strongly-augmented views of an image,

Masked Siamese Networks for Label-Efficient Learning

TLDR
This work proposes Masked Siamese Networks (MSN), a self-supervised learning framework for learning image representations that improves the scalability of joint-embedding architectures, while producing representations of a high semantic level that perform competitively on low-shot image classification.

Building One-Shot Semi-Supervised (BOSS) Learning Up to Fully Supervised Performance

TLDR
This work demonstrates for the first time the potential for building one-shot semi-supervised (BOSS) learning on CIFAR-10 and SVHN up to attain test accuracies that are comparable to fully supervised learning.

References

SHOWING 1-10 OF 47 REFERENCES

Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning

TLDR
This work shows that a naive pseudo-labeling overfits to incorrect pseudo-labels due to the so-called confirmation bias and demonstrates that mixup augmentation and setting a minimum number of labeled samples per mini-batch are effective regularization techniques for reducing it.

Unsupervised Data Augmentation for Consistency Training

TLDR
A new perspective on how to effectively noise unlabeled examples is presented and it is argued that the quality of noising, specifically those produced by advanced data augmentation methods, plays a crucial role in semi-supervised learning.

S4L: Self-Supervised Semi-Supervised Learning

TLDR
It is shown that S4L and existing semi-supervised methods can be jointly trained, yielding a new state-of-the-art result on semi- supervised ILSVRC-2012 with 10% of labels.

Boosting the Performance of Semi-Supervised Learning with Unsupervised Clustering

TLDR
It is shown that ignoring the labels altogether for whole epochs intermittently during training can significantly improve performance in the small sample regime, and the method's efficacy in boosting several state-of-the-art SSL algorithms is demonstrated.

Big Self-Supervised Models are Strong Semi-Supervised Learners

TLDR
The proposed semi-supervised learning algorithm can be summarized in three steps: unsupervised pretraining of a big ResNet model using SimCLRv2 (a modification of SimCLRs), supervised fine-tuning on a few labeled examples, and distillation with unlabeled examples for refining and transferring the task-specific knowledge.

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

TLDR
This paper demonstrates the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling, and shows that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks.

CoMatch: Semi-supervised Learning with Contrastive Graph Regularization

TLDR
CoMatch is a new semi-supervised learning method that unifies dominant approaches and addresses their limitations, and achieves substantial accuracy improvements on the label-scarce CIFAR-10 and STL-10.

Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning

TLDR
An unsupervised loss function is proposed that takes advantage of the stochastic nature of these methods and minimizes the difference between the predictions of multiple passes of a training sample through the network.

Semi-Supervised Self-Training of Object Detection Models

TLDR
The key contributions of this empirical study are to demonstrate that a model trained in this manner can achieve results comparable to a modeltrained in the traditional manner using a much larger set of fully labeled data, and that a training data selection metric that is defined independently of the detector greatly outperforms a selection metric based on the detection confidence generated by the detector.

Density-Aware Graph for Deep Semi-Supervised Visual Recognition

TLDR
This paper proposes a new Density-aware Neighborhood Aggregation module to learn more discriminative features by incorporating the neighborhood information in a density-aware manner and a novel D density-ascending Path based Label Propagation module to generate the pseudo-labels for unlabeled samples more efficiently according to the feature distribution characterized by density.