Corpus ID: 230799353

Distribution-Free, Risk-Controlling Prediction Sets

@article{Bates2021DistributionFreeRP,
  title={Distribution-Free, Risk-Controlling Prediction Sets},
  author={Stephen Bates and A. Angelopoulos and Lihua Lei and J. Malik and Michael I. Jordan},
  journal={ArXiv},
  year={2021},
  volume={abs/2101.02703}
}
To communicate instance-wise uncertainty for prediction tasks, we show how to generate set-valued predictions for black-box predictors that control the expected loss on future test points at a user-specified level. Our approach provides explicit finite-sample guarantees for any dataset by using a holdout set to calibrate the size of the prediction sets. This framework enables simple, distribution-free, rigorous error control for many tasks, and we demonstrate it in five large-scale machine… Expand
4 Citations
Private Prediction Sets
  • PDF
Few-shot Conformal Prediction with Auxiliary Tasks
  • 1
  • PDF
Consistent Accelerated Inference via Confident Adaptive Transformers
  • PDF
Testing for Outliers with Conformal p-values
  • PDF

References

SHOWING 1-10 OF 61 REFERENCES
Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction
  • 7
  • PDF
Least Ambiguous Set-Valued Classifiers With Bounded Error Levels
  • 32
  • Highly Influential
  • PDF
PAC Confidence Predictions for Deep Neural Network Classifiers
  • 2
  • PDF
Cautious Deep Learning
  • 20
  • PDF
Nested conformal prediction and quantile out-of-bag ensemble methods
  • 12
  • Highly Influential
  • PDF
...
1
2
3
4
5
...