• Corpus ID: 225062556

# Coping with Label Shift via Distributionally Robust Optimisation

@article{Zhang2020CopingWL,
title={Coping with Label Shift via Distributionally Robust Optimisation},
author={J. Zhang and Aditya Krishna Menon and Andreas Veit and Srinadh Bhojanapalli and Sanjiv Kumar and Suvrit Sra},
journal={ArXiv},
year={2020},
volume={abs/2010.12230}
}
• Published 23 October 2020
• Computer Science
• ArXiv
The label shift problem refers to the supervised learning setting where the train and test label distributions do not match. Existing work addressing label shift usually assumes access to an \emph{unlabelled} test sample. This sample may be used to estimate the test label distribution, and to then train a suitably re-weighted classifier. While approaches using this idea have proven effective, their scope is limited as it is not always feasible to access the target domain; further, they require…
27 Citations

## Figures and Tables from this paper

• Computer Science
NeurIPS
• 2021
This paper introduces a modeling framework where, in addition to training data, the author has partial structural knowledge of the shifted test distribution, and employs the principle of minimum discriminating information to embed the available prior knowledge.
• Computer Science
ICML
• 2021
This paper proposes a simple two-stage approach, JTT, that minimizes the loss over a reweighted dataset where the authors upweight training examples that are misclassified at the end of a few steps of standard training, leading to improved worst-group performance.
• Computer Science
ArXiv
• 2022
Stochastic algorithms for a class of DRO problems including group DRO, subpopulation fairness, and empirical conditional value at risk (CVaR) optimization are devised.
• Computer Science
ArXiv
• 2022
A framework for ranking images within their class based on the strength of spurious cues present and a second metric to assess feature reliability, finding that spurious features are generally less reliable than non-spurious ones, though again, spurious features can be more reliable for certain classes.
• Computer Science
ArXiv
• 2022
A novel LNL method to simultaneously deal with noisy labels and imbalanced subpopulations is proposed, which leverages sample correlation to estimate samples’ clean probabilities for label correction and then utilizes corrected labels for Distributionally Robust Optimization (DRO) to further improve the robustness.
• Computer Science
• 2022
This paper introduces RLS BENCH, a large-scale benchmark for such relaxed label shift settings, and develops an effective meta-algorithm, compatible with most deep domain adaptation heuristics, that consists of the following two steps: pseudo-balance the data at each epoch and adjust the final class with an estimate of a target label distribution.
• Computer Science
ArXiv
• 2022
This work proposes to unify and generalize self-supervision and oversampling using a class-conditional variant of mixup tailored for worst-group generalization, and shows that this approach, Just Mix Once (JM1), performs on par or better than the state-of-the-art on worst- group generalization.
• Computer Science
ArXiv
• 2022
Wild-Time is curate, a benchmark of 7 datasets that reflect temporal distribution shifts arising in a variety of real-world applications that demonstrate that existing methods are limited in tackling temporal distribution shift.
• Computer Science
ArXiv
• 2022
It is shown that adversarially trained classiﬁers rely on backgrounds more than their standardly trained counterparts, validating the theoretical results and revealing that previous claims that adversarial vulnerability is rooted in spurious correlations are incomplete.

## References

SHOWING 1-10 OF 52 REFERENCES

• Computer Science
ICLR
• 2019
We propose Regularized Learning under Label shifts (RLLS), a principled and a practical domain-adaptation algorithm to correct for shifts in the label distribution between a source and a target
In machine learning, if the training data is an unbiased sample of an underlying distribution, then the learned classification function will make accurate predictions for new samples. However, if the
• Computer Science
NIPS
• 2006
The main advantages of this new formulation over previous models for covariate shift are that the test and training densities are known, the regression and density estimation are combined into a single procedure, and previous methods are reproduced as special cases of this procedure, shedding light on the implicit assumptions the methods are making.
• Computer Science
NeurIPS
• 2020
A recent upper-bound on the performance of adversarial domain adaptation is extended to multi-class classification and more general discriminators and generalized label shift (GLS) is proposed as a way to improve robustness against mismatched label distributions.
• Computer Science
ICML
• 2018
This paper proves that the DRSL just ends up giving a classifier that exactly fits the given training distribution, which is too pessimistic, and proposes simple D RSL that overcomes this pessimism and empirically demonstrate its effectiveness.
• Computer Science
NeurIPS
• 2020
A unified view of the two methods and the first theoretical characterization of the likelihood-based estimator is presented, attributing BBSE's statistical inefficiency to a loss of information due to coarse calibration.
• Computer Science
ICML
• 2018
Black Box Shift Estimation (BBSE) is proposed to estimate the test distribution of p(y) and it is proved BBSE works even when predictors are biased, inaccurate, or uncalibrated, so long as their confusion matrices are invertible.
• Computer Science
ICML
• 2013
This work considers domain adaptation under three possible scenarios, kernel embedding of conditional as well as marginal distributions, and proposes to estimate the weights or transformations by reweighting or transforming training data to reproduce the covariate distribution on the test domain.
• Computer Science
NeurIPS
• 2019
A theoretically-principled label-distribution-aware margin (LDAM) loss motivated by minimizing a margin-based generalization bound is proposed that replaces the standard cross-entropy objective during training and can be applied with prior strategies for training with class-imbalance such as re-weighting or re-sampling.
• Computer Science
ICLR
• 2020
It is shown that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.