• Corpus ID: 231839750

Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

@inproceedings{Cherubin2021ExactOO,
  title={Exact Optimization of Conformal Predictors via Incremental and Decremental Learning},
  author={Giovanni Cherubin and Konstantinos Chatzikokolakis and Martin Jaggi},
  booktitle={ICML},
  year={2021}
}
Conformal Predictors (CP) are wrappers around ML models, providing error guarantees under weak assumptions on the data distribution. They are suitable for a wide range of problems, from classification and regression to anomaly detection. Unfortunately, their very high computational complexity limits their applicability to large datasets. In this work, we show that it is possible to speed up a CP classifier considerably, by studying it in conjunction with the underlying ML method, and by… 
2 Citations

Figures and Tables from this paper

Approximating Full Conformal Prediction at Scale via Influence Functions
TLDR
This paper proves that their method is a consistent approximation of full CP, and empirically shows that the approximation error becomes smaller as the training set increases; e.g., for 10 training points the two methods output p-values that are < 10−3 apart: a negligible error for any practical application.
Fast conformal classification using influence functions
TLDR
This work uses influence functions from robust statistics to speed up full conformal prediction, which requires retraining multiple leave-one-out classifiers to calculate p-values for each test point.

References

SHOWING 1-10 OF 30 REFERENCES
Regression Conformal Prediction with Nearest Neighbours
TLDR
This paper defines six novel nonconformity measures based on the k-Nearest Neighbours Regression algorithm and develops the corresponding CPs following both the original (transductive) and the inductive CP approaches.
Exact Incremental and Decremental Learning for LS-SVM
TLDR
A provably exact method to adapt a pre-trained model to changes in the training dataset, without retraining the model on all the data, where the changes can include addition and deletion of data samples.
Algorithmic Learning in a Random World
TLDR
A selection of books about type systems in programming languages, information theory, and machine learning that takes the randomness of the world into account, and verification of real time systems.
Testing Exchangeability On-Line
TLDR
The notion of exchangeability martingales is introduced, which are online procedures for detecting deviations from exchangeability; in essence, they are betting schemes that never risk bankruptcy and are fair under the hypothesis of exchangeable.
Ridge Regression Confidence Machine
Predictive inference with the jackknife+
This paper introduces the jackknife+, which is a novel method for constructing predictive confidence intervals. Whereas the jackknife outputs an interval centered at the predicted response of a test
Cross-conformal predictors
  • V. Vovk
  • Chemistry
    Annals of Mathematics and Artificial Intelligence
  • 2013
TLDR
The method of cross-conformal prediction is introduced, which is a hybrid of the methods of inductive conformal prediction and cross-validation, and its validity and informational efficiency empirically are studied.
Nested conformal prediction and quantile out-of-bag ensemble methods
Computing Full Conformal Prediction Set with Approximate Homotopy
TLDR
This work proposes efficient algorithms to compute conformal prediction set using approximated solution of (convex) regularized empirical risk minimization using a new homotopy continuation technique for tracking the solution path with respect to sequential changes of the observations.
...
1
2
3
...