An introduction to ROC analysis

@article{Fawcett2006AnIT,
  title={An introduction to ROC analysis},
  author={Tom Fawcett},
  journal={Pattern Recognit. Lett.},
  year={2006},
  volume={27},
  pages={861-874}
}
  • Tom Fawcett
  • Published 1 June 2006
  • Computer Science
  • Pattern Recognit. Lett.

Evaluating Classifiers Using ROC Curves

This work surveys ROC analysis, highlighting the advantages of its use in machine learning and data mining, and tries to clarify several erroneous interpretations related to its use.

Decision Making with Machine Learning and ROC Curves

The implications of different models of incentive heterogeneity and information asymmetry on the relation between human decisions and the ROC curves are analyzed.

Comparing Empirical ROC Curves Using a Java Application: CERCUS

Receiver Operating Characteristic (ROC) analysis is a methodology that has gained much popularity in our days, especially in Medicine, since through the ROC curves, it provides a useful tool to

Multi-objective ROC learning for classification

A multi-objective evolutionary algorithm (MOEA) is used to find classifiers whose ROC graph locations are Pareto optimal, used to identify genes for classification by simultaneously maximising the area under the ROC curve whilst minimising model complexity.

ROC analysis of classifiers in machine learning: A survey

A survey of the application areas of the ROC analysis in machine learning is presented, its problems and challenges are described and a summarized list of alternative approaches to ROCAnalysis is provided.

Measuring Classifier Performance with Risk and Error Matrix Charts

Challenges with using risk charts and error matrix charts including how base rates and prevalence data for building models and incidence data for evaluating models affect model performance are discussed.

Heckroccurve: ROC Curves for Selected Samples

Receiver operating characteristic (ROC) curves can be misleading when they are constructed with selected samples. In this article, we describe heckroccurve, which implements a recently developed

Empirical Comparison of Area under ROC curve (AUC) and Mathew Correlation Coefficient (MCC) for Evaluating Machine Learning Algorithms on Imbalanced Datasets for Binary Classification

This study utilizes an earlier-proposed criteria for comparing metrics based on the degree of consistency and degree of Discriminancy to compare AUC against Matthew Correlation Coefficient and demonstrates that both AUC and MCC are statistically consistent with each other; however AUC is more discriminating than MCC.

Receiver Operating Characteristic (ROC) Analysis

The Receiver Operating Characteristic (ROC) is widely applied to assess the performance of spatial models that produce probability maps of the occurrence of certain events such as the land use / land

On the scalability of ordered multi-class ROC analysis

...

References

SHOWING 1-10 OF 32 REFERENCES

The use of the area under the ROC curve in the evaluation of machine learning algorithms

Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions

The ROC convex hull method combines techniques from ROC analysis, decision analysis and computational geometry, and adapts them to the particulars of analyzing learned classifiers to present a method for the comparison of classifier performance that is robust to imprecise class distributions and misclassification costs.

The Case against Accuracy Estimation for Comparing Induction Algorithms

This work describes and demonstrates what it believes to be the proper use of ROC analysis for comparative studies in machine learning research, and argues that this methodology is preferable both for making practical choices and for drawing conclusions.

Classification and Regression Trees

This chapter discusses tree classification in the context of medicine, where right Sized Trees and Honest Estimates are considered and Bayes Rules and Partitions are used as guides to optimal pruning.

Using rule sets to maximize ROC performance

  • Tom Fawcett
  • Computer Science
    Proceedings 2001 IEEE International Conference on Data Mining
  • 2001
This paper empirically investigates different strategies for evaluating rule sets when the goal is to maximize the scoring (ROC) performance.

Confidence Bands for ROC Curves: Methods and an Empirical Study

It is shown how some of these methods for generating and evaluating condence bands on ROC curves work remarkably well, others are too loose, and that existing machine learning methods for generation of 1-dimensional condence intervals do not translate well to generation of simultanous bands|their bands are too tight.

Extensions of ROC Analysis to multi-class domains

Receiver-operating characteristic (ROC) analysis has proven to be a powerful method for dealing with misclassification costs and skewed class distributions (Provost & Fawcett, 1998). In the typical

The meaning and use of the area under a receiver operating characteristic (ROC) curve.

A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient characteristics,

Measuring the accuracy of diagnostic systems.

For diagnostic systems used to distinguish between two classes of events, analysis in terms of the "relative operating characteristic" of signal detection theory provides a precise and valid measure of diagnostic accuracy.

Combining Data Mining and Machine Learning for Effective User Profiling

This paper combines data mining and constructive induction with more standard machine learning techniques to design methods for detecting fraudulent usage of cellular telephones based on profiling customer behavior, and uses a rule-learning program to uncover indicators of fraudulent behavior from a large database of cellular calls.