• Corpus ID: 236987301

How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers

  title={How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers},
  author={Marharyta Aleksandrova and Oleg G. Chertov},
The property of conformal predictors to guarantee the required accuracy rate makes this framework attractive in various practical applications. However, this property is achieved at a price of reduction in precision. In the case of conformal classification, the systems can output multiple class labels instead of one. It is also known from the literature, that the choice of nonconformity function has a major impact on the efficiency of conformal classifiers. Recently, it was shown that different… 



Model-agnostic nonconformity functions for conformal classification

Three model-agnostic nonconformity functions, based on well-known loss functions, are evaluated with regard to how they affect efficiency on 21 publicly available multi-class data sets, showing that the choice of non Conformity function has a major impact on the efficiency, but also that different nonconforming functions should be used depending on the exact efficiency metric.

Finite-sample Efficient Conformal Prediction

This paper provides two general-purpose selection algorithms and considers coverage as well as width properties of the final prediction region of the conformal prediction regions given a family of machine learning algorithms.

Synergy conformal prediction

The synergy conformal predictor is introduced which is an ensemble method that combines monotonic conformity scores, and is capable of producing valid prediction intervals that on partitioned data performs well compared to the most efficient model trained on individual partitions.

A tutorial on conformal prediction

This tutorial presents a self-contained account of the theory of conformal prediction and works through several numerical examples of how the model under which successive examples are sampled independently from the same distribution can be applied to any method for producing ŷ.

Efficient conformal predictor ensembles

Combination of Conformal Predictors for Classification

The experiments show that P-value combination, in particular Fisher’s method, can be advantageous when ranking compounds by strength of evidence.

Support vector machine based conformal predictors for risk of complications following a coronary Drug Eluting Stent procedure

This work has applied a novel Support Vector Machine based conformal prediction framework to predict the risk of complications following a coronary DES procedure, and this predictive model helps to risk stratify a patient for post-DES complications.

Transductive conformal predictors

This paper discusses a transductive version of conformal predictors. This version is computationally inefficient for big test sets, but it turns out that apparently crude “Bonferroni predictors” are

Majority vote ensembles of conformal predictors

An error upper bound for an ensemble of correlated CPs is indicated, and a value is derived for which such an ensemble guarantees conservative validity is derived.

Algorithmic Learning in a Random World

A selection of books about type systems in programming languages, information theory, and machine learning that takes the randomness of the world into account, and verification of real time systems.