Evaluating Classifiers Using ROC Curves
- Computer ScienceIEEE Latin America Transactions
This work surveys ROC analysis, highlighting the advantages of its use in machine learning and data mining, and tries to clarify several erroneous interpretations related to its use.
Decision Making with Machine Learning and ROC Curves
- Computer ScienceSSRN Electronic Journal
The implications of different models of incentive heterogeneity and information asymmetry on the relation between human decisions and the ROC curves are analyzed.
Comparing Empirical ROC Curves Using a Java Application: CERCUS
- Computer ScienceICCSA
Receiver Operating Characteristic (ROC) analysis is a methodology that has gained much popularity in our days, especially in Medicine, since through the ROC curves, it provides a useful tool to…
ROC analysis of classifiers in machine learning: A survey
- Computer ScienceIntell. Data Anal.
A survey of the application areas of the ROC analysis in machine learning is presented, its problems and challenges are described and a summarized list of alternative approaches to ROCAnalysis is provided.
Measuring Classifier Performance with Risk and Error Matrix Charts
- BusinessTrans. Mach. Learn. Data Min.
Challenges with using risk charts and error matrix charts including how base rates and prevalence data for building models and incidence data for evaluating models affect model performance are discussed.
Heckroccurve: ROC Curves for Selected Samples
Receiver operating characteristic (ROC) curves can be misleading when they are constructed with selected samples. In this article, we describe heckroccurve, which implements a recently developed…
Empirical Comparison of Area under ROC curve (AUC) and Mathew Correlation Coefficient (MCC) for Evaluating Machine Learning Algorithms on Imbalanced Datasets for Binary Classification
- Computer ScienceICMLSC 2019
This study utilizes an earlier-proposed criteria for comparing metrics based on the degree of consistency and degree of Discriminancy to compare AUC against Matthew Correlation Coefficient and demonstrates that both AUC and MCC are statistically consistent with each other; however AUC is more discriminating than MCC.
Receiver Operating Characteristic (ROC) Analysis
The Receiver Operating Characteristic (ROC) is widely applied to assess the performance of spatial models that produce probability maps of the occurrence of certain events such as the land use / land…
On the scalability of ordered multi-class ROC analysis
- Computer ScienceComput. Stat. Data Anal.
SHOWING 1-10 OF 32 REFERENCES
The use of the area under the ROC curve in the evaluation of machine learning algorithms
- Computer SciencePattern Recognit.
Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions
- Computer ScienceKDD
The ROC convex hull method combines techniques from ROC analysis, decision analysis and computational geometry, and adapts them to the particulars of analyzing learned classifiers to present a method for the comparison of classifier performance that is robust to imprecise class distributions and misclassification costs.
The Case against Accuracy Estimation for Comparing Induction Algorithms
- Computer ScienceICML
This work describes and demonstrates what it believes to be the proper use of ROC analysis for comparative studies in machine learning research, and argues that this methodology is preferable both for making practical choices and for drawing conclusions.
Classification and Regression Trees
- Computer Science
This chapter discusses tree classification in the context of medicine, where right Sized Trees and Honest Estimates are considered and Bayes Rules and Partitions are used as guides to optimal pruning.
Using rule sets to maximize ROC performance
- Computer ScienceProceedings 2001 IEEE International Conference on Data Mining
This paper empirically investigates different strategies for evaluating rule sets when the goal is to maximize the scoring (ROC) performance.
Confidence Bands for ROC Curves: Methods and an Empirical Study
- Computer ScienceROCAI
It is shown how some of these methods for generating and evaluating condence bands on ROC curves work remarkably well, others are too loose, and that existing machine learning methods for generation of 1-dimensional condence intervals do not translate well to generation of simultanous bands|their bands are too tight.
Extensions of ROC Analysis to multi-class domains
- Computer Science
Receiver-operating characteristic (ROC) analysis has proven to be a powerful method for dealing with misclassification costs and skewed class distributions (Provost & Fawcett, 1998). In the typical…
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient characteristics,…
Measuring the accuracy of diagnostic systems.
- Computer ScienceScience
For diagnostic systems used to distinguish between two classes of events, analysis in terms of the "relative operating characteristic" of signal detection theory provides a precise and valid measure of diagnostic accuracy.
Combining Data Mining and Machine Learning for Effective User Profiling
- Computer ScienceKDD
This paper combines data mining and constructive induction with more standard machine learning techniques to design methods for detecting fraudulent usage of cellular telephones based on profiling customer behavior, and uses a rule-learning program to uncover indicators of fraudulent behavior from a large database of cellular calls.