• Corpus ID: 166227922

Combating Label Noise in Deep Learning Using Abstention

@inproceedings{Thulasidasan2019CombatingLN,
  title={Combating Label Noise in Deep Learning Using Abstention},
  author={Sunil Thulasidasan and Tanmoy Bhattacharya and Jeff A. Bilmes and Gopinath Chennupati and Jamaludin Mohd-Yusof},
  booktitle={ICML},
  year={2019}
}
We introduce a novel method to combat label noise when training deep neural networks for classification. [...] Key Method In the case of unstructured (arbitrary) label noise, abstention during training enables the DAC to be used as an effective data cleaner by identifying samples that are likely to have label noise. We provide analytical results on the loss function behavior that enable dynamic adaption of abstention rates based on learning progress during training. We demonstrate the utility of the deep…Expand
SELF: Learning to Filter Noisy Labels with Self-Ensembling
TLDR
This work presents a simple and effective method self-ensemble label filtering (SELF) to progressively filter out the wrong labels during training that substantially outperforms all previous works on noise-aware learning across different datasets and can be applied to a broad set of network architectures.
Confidence Adaptive Regularization for Deep Learning with Noisy Labels
TLDR
This work introduces an indicator branch to the original model and enables the model to produce a confidence value for each sample, and proposes an auxiliary regularization term to further improve the robustness of the model.
A Survey on Deep Learning with Noisy Labels: How to train your model when you cannot trust on the annotations?
  • F. Cordeiro, G. Carneiro
  • Computer Science
    2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI)
  • 2020
TLDR
A survey on the main techniques in literature to improve the training of deep learning models in the presence of noisy labels is presented, in which the algorithm is classified in the following groups: robust losses, sample weighting, sample selection, meta-learning, and combined approaches.
Generalization by Recognizing Confusion
TLDR
It is found that the Rademacher complexity of these algorithms is low, suggesting a new path towards provable generalization for this type of deep learning model, and highlights a novel connection between difficulties accounting for rare classes and robustness under noise.
Decoupling Representation and Classifier for Noisy Label Learning
TLDR
It is discovered that the representation is much more fragile in the presence of noisy labels than the classifier, and a new method, i.e., REED, is designed to leverage above discoveries to learn from noisy labels robustly.
Does label smoothing mitigate label noise?
TLDR
It is shown that when distilling models from noisy data,label smoothing of the teacher is beneficial; this is in contrast to recent findings for noise-free problems, and sheds further light on settings where label smoothing is beneficial.
Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
TLDR
This work proposes a robust and efficient method that learns a label transition matrix on the fly that makes the classifier skeptical about all the corrected samples, which alleviates the miscorrection issue.
DivideMix: Learning with Noisy Labels as Semi-supervised Learning
TLDR
This work proposes DivideMix, a novel framework for learning with noisy labels by leveraging semi-supervised learning techniques, which models the per-sample loss distribution with a mixture model to dynamically divide the training data into a labeled set with clean samples and an unlabeled set with noisy samples.
INN: A Method Identifying Clean-annotated Samples via Consistency Effect in Deep Neural Networks
TLDR
The INN method is introduced, based on a new discovery that a prediction pattern at neighbor regions of clean labeled data is consistently different from that of noisy labeled data regardless of training epochs, which is helpful to construct more accurate deep prediction models with training data with noisy labels.
Error-Bounded Correction of Noisy Labels
TLDR
It is proved that the prediction of a noisy classifiers can indeed be a good indicator of whether the label of a training data is clean and a novel algorithm is proposed that corrects the labels based on the noisy classifier prediction.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 43 REFERENCES
Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
TLDR
It is demonstrated that robustness to label noise up to severe strengths can be achieved by using a set of trusted data with clean labels, and a loss correction that utilizes trusted examples in a data-efficient manner to mitigate the effects of label noise on deep neural network classifiers is proposed.
Training Deep Neural Networks on Noisy Labels with Bootstrapping
TLDR
A generic way to handle noisy and incomplete labeling by augmenting the prediction objective with a notion of consistency is proposed, which considers a prediction consistent if the same prediction is made given similar percepts, where the notion of similarity is between deep network features computed from the input data.
Learning from Noisy Labels with Distillation
TLDR
This work proposes a unified distillation framework to use “side” information, including a small clean dataset and label relations in knowledge graph, to “hedge the risk” of learning from noisy labels, and proposes a suite of new benchmark datasets to evaluate this task in Sports, Species and Artifacts domains.
Deep Learning is Robust to Massive Label Noise
TLDR
It is shown that deep neural networks are capable of generalizing from training data for which true labels are massively outnumbered by incorrect labels, and that training in this regime requires a significant but manageable increase in dataset size that is related to the factor by which correct labels have been diluted.
Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
TLDR
A theoretically grounded set of noise-robust loss functions that can be seen as a generalization of MAE and CCE are presented and can be readily applied with any existing DNN architecture and algorithm, while yielding good performance in a wide range of noisy label scenarios.
Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach
TLDR
It is proved that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise, and it is shown how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and providing an end-to-end framework.
MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
TLDR
Experimental results demonstrate that the proposed novel technique of learning another neural network, called MentorNet, to supervise the training of the base deep networks, namely, StudentNet, can significantly improve the generalization performance of deep networks trained on corrupted training data.
Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks
TLDR
The proposed novel framework for training deep convolutional neural networks from noisy labeled datasets that can be obtained cheaply is applied to the image labeling problem and is shown to be effective in labeling unseen images as well as reducing label noise in training on CIFAR-10 and MS COCO datasets.
Co-teaching: Robust training of deep neural networks with extremely noisy labels
TLDR
Empirical results on noisy versions of MNIST, CIFar-10 and CIFAR-100 demonstrate that Co-teaching is much superior to the state-of-the-art methods in the robustness of trained deep models.
Learning and Evaluation in Presence of Non-i.i.d. Label Noise
TLDR
This paper presents a novel methodology for learning and evaluation in presence of systematic label noise, the core of which is a novel extension of support vector data description / one-class SVM that can incorporate latent variables.
...
1
2
3
4
5
...