# Finite-sample Efficient Conformal Prediction

@inproceedings{Yang2021FinitesampleEC, title={Finite-sample Efficient Conformal Prediction}, author={Yachong Yang and Arun K. Kuchibhotla}, year={2021} }

Conformal prediction is a generic methodology for finite-sample valid distribution-free prediction. This technique has garnered a lot of attention in the literature partly because it can be applied with any machine learning algorithm that provides point predictions to yield valid prediction regions. Of course, the efficiency (width/volume) of the resulting prediction region depends on the performance of the machine learning algorithm. In this paper, we consider the problem of obtaining the…

## 19 Citations

### Improved Online Conformal Prediction via Strongly Adaptive Online Learning

- Computer Science
- 2023

New online conformal prediction methods are developed that minimize the strongly adaptive regret, which measures the worst-case regret over all intervals of a fixed length, and it is proved that these methods consistently obtain better coverage and smaller prediction sets than existing methods on real-world tasks.

### Exchangeability, Conformal Prediction, and Rank Tests

- Computer Science
- 2020

The main message of the paper is to show that similar to conformal prediction, rank tests can also be used as a wrapper around any dimension reduction algorithm.

### Efficient and Differentiable Conformal Prediction with General Function Classes

- Computer ScienceICLR
- 2022

This meta-algorithm generalizes existing conformal prediction algorithms, and it achieves approximate valid population coverage and near-optimal eﬃciency within class, whenever the function class in the conformalization step is low-capacity in a certain sense.

### Impact of model-agnostic nonconformity functions on efficiency of conformal classifiers: an extensive study

- Computer ScienceCOPA
- 2021

This work performs an experimental evaluation using 8 different classification algorithms and proposes a successful method to combine the properties of these two nonconformity functions.

### How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers

- Computer ScienceArXiv
- 2021

This work performs an experimental evaluation using 8 different classification algorithms and discusses when the previously observed relationship holds or not, and proposes a successful method to combine the properties of these two nonconformity functions.

### Conformal Prediction using Conditional Histograms

- Computer ScienceNeurIPS
- 2021

A conformal method to compute prediction intervals for nonparametric regression that can automatically adapt to skewed data and have marginal coverage in finite samples, while asymptotically achieving conditional coverage and optimal length if the black-box model is consistent.

### Conformal histogram regression

- Computer Science
- 2021

A conformal method to compute prediction intervals for nonparametric regression that can automatically adapt to skewed data and have marginal coverage in finite samples, while asymptotically achieving conditional coverage and optimal length if the black-box model is consistent.

### Predictive Inference with Feature Conformal Prediction

- Computer ScienceArXiv
- 2022

This paper proposes feature conformal Prediction, which extends the scope of conformal prediction to semantic feature spaces by leveraging the inductive bias of deep representation learning and demonstrates that feature conformAL prediction provably outperforms regular conformal predicted intervals under mild assumptions.

### Training-conditional coverage for distribution-free predictive inference

- Computer Science
- 2022

This work examines the training-conditional coverage properties of several distribution-free predictive inference methods and concludes that training-Conditional coverage is achieved by some methods but is impossible to guarantee without further assumptions for others.

### Conformalized Survival Analysis.

- MathematicsJournal of the Royal Statistical Society Series B: Statistical Methodology
- 2023

This paper develops an inferential method based on ideas from conformal prediction, which can wrap around any survival prediction algorithm to produce calibrated, covariate-dependent lower predictive bounds on survival times.

## References

SHOWING 1-10 OF 57 REFERENCES

### Distribution-Free Predictive Inference for Regression

- Computer Science, MathematicsJournal of the American Statistical Association
- 2018

A general framework for distribution-free predictive inference in regression, using conformal inference, which allows for the construction of a prediction band for the response variable using any estimator of the regression function, and a model-free notion of variable importance, called leave-one-covariate-out or LOCO inference.

### Efficiency of conformalized ridge regression

- Computer ScienceCOLT
- 2014

The degree to which this additional requirement of efficiency is satisfied in the case of Bayesian ridge regression is explored; it is found that asymptotically conformal prediction sets differ little from ridge regression prediction intervals when the standard Bayesian assumptions are satisfied.

### Nested conformal prediction and quantile out-of-bag ensemble methods

- Computer SciencePattern Recognit.
- 2022

### CD-split: efficient conformal regions in high dimensions

- Computer ScienceArXiv
- 2020

It is shown that CD-split converges asymptotically to the oracle highest density set and satisfies local and asymPTotic conditional validity, and has a better conditional coverage and yields smaller prediction regions than other methods.

### Regression conformal prediction with random forests

- Computer ScienceMachine Learning
- 2014

In this study, the use of random forests as the underlying model for regression conformal prediction is investigated and compared to existing state-of-the-art techniques, which are based on neural networks and k-nearest neighbors.

### A comparison of some conformal quantile regression methods

- MathematicsStat
- 2020

We compare two recent methods that combine conformal inference with quantile regression to produce locally adaptive and marginally valid prediction intervals under sample exchangeability (Romano,…

### Conformalized Quantile Regression

- Computer Science, MathematicsNeurIPS
- 2019

This paper proposes a new method that is fully adaptive to heteroscedasticity, which combines conformal prediction with classical quantile regression, inheriting the advantages of both.

### Distribution-Free Prediction Sets

- MathematicsJournal of the American Statistical Association
- 2013

This article considers the problem of constructing nonparametric tolerance/prediction sets by starting from the general conformal prediction approach, and uses a kernel density estimator as a measure of agreement between a sample point and the underlying distribution.

### Nearest neighbor based conformal prediction

- Computer Science, Mathematics
- 2020

A nearest neighbor based estimate of the prediction interval with prescribed conditional coverage probability and with small length is introduced and a confidence set estimate for classification is constructed.

### Distribution‐free prediction bands for non‐parametric regression

- Mathematics, Computer Science
- 2014

A new prediction band is given by combining the idea of ‘conformal prediction’ with non‐parametric conditional density estimation and the proposed estimator, called COPS, always has a finite sample guarantee.