• Corpus ID: 233423342

Finite-sample Efficient Conformal Prediction

@inproceedings{Yang2021FinitesampleEC,
  title={Finite-sample Efficient Conformal Prediction},
  author={Yachong Yang and Arun K. Kuchibhotla},
  year={2021}
}
Conformal prediction is a generic methodology for finite-sample valid distribution-free prediction. This technique has garnered a lot of attention in the literature partly because it can be applied with any machine learning algorithm that provides point predictions to yield valid prediction regions. Of course, the efficiency (width/volume) of the resulting prediction region depends on the performance of the machine learning algorithm. In this paper, we consider the problem of obtaining the… 
[PDF] Semantic Reader
19 Citations
Highly Influential Citations
4
Background Citations
9
Methods Citations
6
Results Citations
1

Figures and Tables from this paper

19 Citations

Improved Online Conformal Prediction via Strongly Adaptive Online Learning

New online conformal prediction methods are developed that minimize the strongly adaptive regret, which measures the worst-case regret over all intervals of a fixed length, and it is proved that these methods consistently obtain better coverage and smaller prediction sets than existing methods on real-world tasks.

Exchangeability, Conformal Prediction, and Rank Tests

The main message of the paper is to show that similar to conformal prediction, rank tests can also be used as a wrapper around any dimension reduction algorithm.

Efficient and Differentiable Conformal Prediction with General Function Classes

This meta-algorithm generalizes existing conformal prediction algorithms, and it achieves approximate valid population coverage and near-optimal efficiency within class, whenever the function class in the conformalization step is low-capacity in a certain sense.

Impact of model-agnostic nonconformity functions on efficiency of conformal classifiers: an extensive study

This work performs an experimental evaluation using 8 different classification algorithms and proposes a successful method to combine the properties of these two nonconformity functions.

How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers

This work performs an experimental evaluation using 8 different classification algorithms and discusses when the previously observed relationship holds or not, and proposes a successful method to combine the properties of these two nonconformity functions.

Conformal Prediction using Conditional Histograms

A conformal method to compute prediction intervals for nonparametric regression that can automatically adapt to skewed data and have marginal coverage in finite samples, while asymptotically achieving conditional coverage and optimal length if the black-box model is consistent.

Conformal histogram regression

A conformal method to compute prediction intervals for nonparametric regression that can automatically adapt to skewed data and have marginal coverage in finite samples, while asymptotically achieving conditional coverage and optimal length if the black-box model is consistent.

Predictive Inference with Feature Conformal Prediction

This paper proposes feature conformal Prediction, which extends the scope of conformal prediction to semantic feature spaces by leveraging the inductive bias of deep representation learning and demonstrates that feature conformAL prediction provably outperforms regular conformal predicted intervals under mild assumptions.

Training-conditional coverage for distribution-free predictive inference

This work examines the training-conditional coverage properties of several distribution-free predictive inference methods and concludes that training-Conditional coverage is achieved by some methods but is impossible to guarantee without further assumptions for others.

Conformalized Survival Analysis.

This paper develops an inferential method based on ideas from conformal prediction, which can wrap around any survival prediction algorithm to produce calibrated, covariate-dependent lower predictive bounds on survival times.

References

SHOWING 1-10 OF 57 REFERENCES

Distribution-Free Predictive Inference for Regression

A general framework for distribution-free predictive inference in regression, using conformal inference, which allows for the construction of a prediction band for the response variable using any estimator of the regression function, and a model-free notion of variable importance, called leave-one-covariate-out or LOCO inference.

Efficiency of conformalized ridge regression

The degree to which this additional requirement of efficiency is satisfied in the case of Bayesian ridge regression is explored; it is found that asymptotically conformal prediction sets differ little from ridge regression prediction intervals when the standard Bayesian assumptions are satisfied.

Nested conformal prediction and quantile out-of-bag ensemble methods

CD-split: efficient conformal regions in high dimensions

It is shown that CD-split converges asymptotically to the oracle highest density set and satisfies local and asymPTotic conditional validity, and has a better conditional coverage and yields smaller prediction regions than other methods.

Regression conformal prediction with random forests

In this study, the use of random forests as the underlying model for regression conformal prediction is investigated and compared to existing state-of-the-art techniques, which are based on neural networks and k-nearest neighbors.

A comparison of some conformal quantile regression methods

We compare two recent methods that combine conformal inference with quantile regression to produce locally adaptive and marginally valid prediction intervals under sample exchangeability (Romano,…

Conformalized Quantile Regression

This paper proposes a new method that is fully adaptive to heteroscedasticity, which combines conformal prediction with classical quantile regression, inheriting the advantages of both.

Distribution-Free Prediction Sets

This article considers the problem of constructing nonparametric tolerance/prediction sets by starting from the general conformal prediction approach, and uses a kernel density estimator as a measure of agreement between a sample point and the underlying distribution.

Nearest neighbor based conformal prediction

A nearest neighbor based estimate of the prediction interval with prescribed conditional coverage probability and with small length is introduced and a confidence set estimate for classification is constructed.

Distribution‐free prediction bands for non‐parametric regression

A new prediction band is given by combining the idea of ‘conformal prediction’ with non‐parametric conditional density estimation and the proposed estimator, called COPS, always has a finite sample guarantee.
...