• Corpus ID: 245425047

Simple and near-optimal algorithms for hidden stratification and multi-group learning

@inproceedings{Tosh2021SimpleAN,
  title={Simple and near-optimal algorithms for hidden stratification and multi-group learning},
  author={Christopher Tosh and Daniel J. Hsu},
  booktitle={International Conference on Machine Learning},
  year={2021}
}
Multi-group agnostic learning is a formal learning criterion that is concerned with the conditional risks of predictors within subgroups of a population. The criterion addresses recent practical concerns such as subgroup fairness and hidden stratification. This paper studies the structure of solutions to the multi-group learning problem, and provides simple and near-optimal algorithms for the learning problem. 
[PDF] Semantic Reader
4 Citations
Highly Influential Citations
1
Background Citations
3
Results Citations
2

Figures from this paper

4 Citations

Beyond the Frontier: Fairness Without Accuracy Loss

A simple algorithmic framework that allows us to deploy models and then revise them dynamically when groups are discovered on which the error rate is suboptimal is developed, and the result is provably fast convergence to a model that cannot be distinguished from the Bayes optimal predictor — at least by the party tasked with finding high error groups.

An Algorithmic Framework for Bias Bounties

An algorithmic framework for “bias bounties” — events in which external participants are invited to propose improvements to a trained model, akin to bug bounty events in software and security, which is algorithmically incorporated into an updated model.

On-Demand Sampling: Learning Optimally from Multiple Distributions

The optimal sample complexity of multi-distribution learning paradigms, such as collaborative, group distributionally robust, and fair federated learning are established and algorithms that meet this sample complexity are given.

Low-Degree Multicalibration

This work defines and initiate the study of Low-Degree Multicalibration, a hierarchy of increasingly-powerful multi-group fairness notions that spans multiaccuracy and the original formulation of multicalsibration at the extremes, and demonstrates that key properties of multicalIBration, related to fairness and accuracy, actually manifest as low-degree properties.

References

SHOWING 1-10 OF 54 REFERENCES

Multi-group Agnostic PAC Learnability

This work unify and extend previous positive and negative results from the multi-group fairness literature, which applied for specific loss functions, to study “multi-group agnostic PAC learnability”.

Equality of Opportunity in Supervised Learning

This work proposes a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features and shows how to optimally adjust any learned predictor so as to remove discrimination according to this definition.

Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization

The results suggest that regularization is important for worst-group generalization in the overparameterized regime, even if it is not needed for average generalization, and introduce a stochastic optimization algorithm, with convergence guarantees, to efficiently train group DRO models.

Distributionally Robust Losses for Latent Covariate Mixtures

The authors propose a convex procedure that controls worst case subpopulation performance and provide finite-sample (nonparametric) convergence guarantees and observe significantly improved performance across unseen subpopulations.

Advancing subgroup fairness via sleeping experts

It is demonstrated that for the simple objective of unweighted average of false negative and false positive rate, satisfying this for overlapping populations can be statistically impossible even when the authors are provided predictors that perform well separately on each subgroup.

Multicalibration: Calibration for the (Computationally-Identifiable) Masses

We develop and study multicalibration as a new measure of fairness in machine learning that aims to mitigate inadvertent or malicious discrimination that is introduced at training time (even from…

Multiaccuracy: Black-Box Post-Processing for Fairness in Classification

It is proved that MULTIACCURACY-BOOST converges efficiently and it is shown that if the initial model is accurate on an identifiable subgroup, then the post-processed model will be also.

On the generalization ability of on-line learning algorithms

This paper proves tight data-dependent bounds for the risk of this hypothesis in terms of an easily computable statistic M/sub n/ associated with the on-line performance of the ensemble, and obtains risk tail bounds for kernel perceptron algorithms interms of the spectrum of the empirical kernel matrix.

Multiaccurate Proxies for Downstream Fairness

This work adopts a fairness pipeline perspective, and shows that obeying multiaccuracy constraints with respect to the downstream model class suffices for this purpose, and provides sample- and oracle efficient-algorithms and generalization bounds for learning such proxies.

Empirical Risk Minimization under Fairness Constraints

This work presents an approach based on empirical risk minimization, which incorporates a fairness constraint into the learning problem, and derives both risk and fairness bounds that support the statistical consistency of the approach.
...