Optimizing ROC Curves with a Sort-Based Surrogate Loss Function for Binary Classification and Changepoint Detection
@article{Hillman2021OptimizingRC, title={Optimizing ROC Curves with a Sort-Based Surrogate Loss Function for Binary Classification and Changepoint Detection}, author={Jonathan A. Hillman and Toby Hocking}, journal={ArXiv}, year={2021}, volume={abs/2107.01285} }
Receiver Operating Characteristic (ROC) curves are plots of true positive rate versus false positive rate which are useful for evaluating binary classification models, but difficult to use for learning since the Area Under the Curve (AUC) is non-convex. ROC curves can also be used in other problems that have false positive and true positive rates such as changepoint detection. We show that in this more general context, the ROC curve can have loops, points with highly sub-optimal error rates…
References
SHOWING 1-10 OF 46 REFERENCES
Optimizing Area Under Roc Curve with SVMs
- Computer ScienceROCAI
- 2004
A quadratic programming based algorithm for AUC maximization is proposed and it is shown that under certain conditions 2-norm soft margin Support Vector Ma- chines can also maximize AUC.
Measuring classifier performance: a coherent alternative to the area under the ROC curve
- Computer ScienceMachine Learning
- 2009
A simple valid alternative to the AUC is proposed, and the property of it being fundamentally incoherent in terms of misclassification costs is explored in detail.
Optimizing area under the ROC curve using semi-supervised learning
- Computer SciencePattern Recognit.
- 2015
A Structural SVM Based Approach for Optimizing Partial AUC
- Computer ScienceICML
- 2013
A structural SVM framework for directly optimizing the partial AUC between any two false positive rates and an efficient algorithm for solving this combinatorial optimization problem that has the same computational complexity as Joachims' algorithm for optimizing the usual AUC is developed.
Learning Decision Trees Using the Area Under the ROC Curve
- Computer ScienceICML
- 2002
This paper shows how a single decision tree can represent a set of classifiers by choosing different labellings of its leaves, or equivalently, an ordering on the leaves, and proposes a novel splitting criterion which chooses the split with the highest local AUC.
Optimising area under the ROC curve using gradient descent
- Computer ScienceICML
- 2004
This paper introduces RankOpt, a linear binary classifier which optimises the area under the ROC curve (the AUC). Unlike standard binary classifiers, RankOpt adopts the AUC statistic as its objective…
AUC Optimization vs. Error Rate Minimization
- Computer ScienceNIPS
- 2003
The results show that the average AUC is monotonically increasing as a function of the classification accuracy, but that the standard deviation for uneven distributions and higher error rates is noticeable, so algorithms designed to minimize the error rate may not lead to the best possible AUC values.
Receiver operating characteristic curve generalization for non-monotone relationships
- MathematicsStatistical methods in medical research
- 2017
A receiver operating characteristic curve generalization, g, is proposed, useful in this context, where both the lower and the upper values of the marker are associated with higher probability of a positive result.
Optimizing Classifier Performance via an Approximation to the Wilcoxon-Mann-Whitney Statistic
- Computer ScienceICML
- 2003
This work proposes an objective function that is an approximation to the Wilcoxon-Mann-Whitney statistic, which is equivalent to the AUC, and applies it to real-world customer behavior prediction problems for a wireless service provider and a cable service provider, and achieves reliable improvements in the ROC curve.
Stochastic Online AUC Maximization
- Computer ScienceNIPS
- 2016
It is shown that AUC optimization can be equivalently formulated as a convex-concave saddle point problem and a stochastic online algorithm (SOLAM) is proposed which has time and space complexity of one datum.