• Corpus ID: 236987301

How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers

@article{Aleksandrova2021HowNF,
  title={How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers},
  author={Marharyta Aleksandrova and Oleg G. Chertov},
  journal={ArXiv},
  year={2021},
  volume={abs/2108.05677}
}
The property of conformal predictors to guarantee the required accuracy rate makes this framework attractive in various practical applications. However, this property is achieved at a price of reduction in precision. In the case of conformal classification, the systems can output multiple class labels instead of one. It is also known from the literature, that the choice of nonconformity function has a major impact on the efficiency of conformal classifiers. Recently, it was shown that different… 

References

SHOWING 1-10 OF 17 REFERENCES
Combination of inductive mondrian conformal predictors
TLDR
This paper proposes and evaluates, alongside methods from Statistical Hypothesis Testing, a heuristically motivated method for learning to combine CPs to improve the quality of prediction, and explores a general nonparametric method for recovering validity after combination using a calibration set.
Synergy conformal prediction
TLDR
The synergy conformal predictor is introduced which is an ensemble method that combines monotonic conformity scores, and is capable of producing valid prediction intervals that on partitioned data performs well compared to the most efficient model trained on individual partitions.
Efficient conformal predictor ensembles
Combination of Conformal Predictors for Classification
TLDR
The experiments show that P-value combination, in particular Fisher’s method, can be advantageous when ranking compounds by strength of evidence.
Inductive Conformal Prediction: Theory and Application to Neural Networks
TLDR
The Bayesian framework and PAC theory can be used for producing upper bounds on the probability of error for a given algorithm with respect to some confidence level 1 − δ; both of these approaches however, have their drawbacks.
Support vector machine based conformal predictors for risk of complications following a coronary Drug Eluting Stent procedure
TLDR
This work has applied a novel Support Vector Machine based conformal prediction framework to predict the risk of complications following a coronary DES procedure, and this predictive model helps to risk stratify a patient for post-DES complications.
Transductive conformal predictors
This paper discusses a transductive version of conformal predictors. This version is computationally inefficient for big test sets, but it turns out that apparently crude “Bonferroni predictors” are
Algorithmic Learning in a Random World
TLDR
A selection of books about type systems in programming languages, information theory, and machine learning that takes the randomness of the world into account, and verification of real time systems.
Transductive Confidence Machines for Pattern Recognition
We propose a new algorithm for pattern recognition that outputs some measures of "reliability" for every prediction made, in contrast to the current algorithms that output "bare" predictions only.
Model-agnostic nonconformity functions for conformal classification
TLDR
Three model-agnostic nonconformity functions, based on well-known loss functions, are evaluated with regard to how they affect efficiency on 21 publicly available multi-class data sets, showing that the choice of non Conformity function has a major impact on the efficiency, but also that different nonconforming functions should be used depending on the exact efficiency metric.
...
...