Using theoretical ROC curves for analysing machine learning binary classifiers

  title={Using theoretical ROC curves for analysing machine learning binary classifiers},
  author={Luma Qassam Abedalqader Omar and Ioannis P. Ivrissimtzis},
  journal={Pattern Recognit. Lett.},
Most binary classifiers work by processing the input to produce a scalar response and comparing it to a threshold value. The various measures of classifier performance assume, explicitly or implicitly, probability distributions $P_s$ and $P_n$ of the response belonging to either class, probability distributions for the cost of each type of misclassification, and compute a performance score from the expected cost. In machine learning, classifier responses are obtained experimentally and… Expand
Multiclass Classification with an Ensemble of Binary Classification Deep Networks
The experiments demonstrate that given sufficient training, a Binary Classification convolutional Neural Network (BCCNN) ensemble can outperform a conventional Multi-class Convolutional neural Network (MCNN). Expand
Extraction of morphological and spectral features of potato plants from high resolution multispectral images
This work studies the use of spectral and morphological features in the evaluation and detection of potato late blight using very high resolution multispectral images captured by Unmanned AerialExpand
Detection and localization enhancement for satellite images with small forgeries using modified GAN-based CNN structure
Two different approaches, for detection and localization of small-sized forgeries in satellite images, are proposed, inspired from a recently presented GAN-based approach and is modified to an enhanced version. Expand
A simulation work for generating a novel dataset to detect distributed denial of service attacks on Vehicular Ad hoc NETwork systems
A novel simulation technique is proposed to generate a valid dataset called Vehicular Ad hoc NETwork distributed denial of service dataset, which is dedicated to Vehicular ad hoc NETworks, and it is confirmed that studied models using this dataset achieved high accuracy above 99.5% except support-vector machine that achieved 97.3%. Expand
Droplets Image Segmentation Method Based on Machine learning and Watershed
Watershed algorithm is used widely in segmentation of droplet overlapped spots on water-sensitive test paper. However, the phenomenon of over-segmentation, however, is often caused by noise andExpand


Measuring classifier performance: a coherent alternative to the area under the ROC curve
  • D. Hand
  • Mathematics, Computer Science
  • Machine Learning
  • 2009
A simple valid alternative to the AUC is proposed, and the property of it being fundamentally incoherent in terms of misclassification costs is explored in detail. Expand
When is the area under the receiver operating characteristic curve an appropriate measure of classifier performance?
If additional information, such as the class assignments of other objects, is taken into account when making a classification, then the area under the curve is a coherent measure, although in those circumstances it makes an assumption which is seldom if ever appropriate. Expand
A better Beta for the H measure of classification performance
This note extends the discussion, and proposes a modified standard distribution for the H measure, which better matches the requirements of researchers, the Beta(@p"1+1, @p"0+1) distribution. Expand
Receiver Operating Characteristic (ROC) Curves
This talk will focus on the use of SAS/STAT procedures FREQ, LOGISTIC, MIXED and NLMIXED to perform ROC analyses, including estimation of sensitivity and specificity, estimation of an ROC curve and computing the area under the ROC curves. Expand
Measuring classification performance : the hmeasure package
The ubiquity of binary classification problems has given rise to a prolific literature dedicated to the proposal of novel classification methods, as well as the incremental improvement of existingExpand
Using Dual Beta Distributions to Create “Proper” ROC Curves Based on Rating Category Data
  • D. Mossman, Hongying Peng
  • Mathematics, Medicine
  • Medical decision making : an international journal of the Society for Medical Decision Making
  • 2016
Because it posits simple relationships among the decision axis, operating points, and model parameters, the DB model offers investigators a flexible, easy-to-grasp ROC form that is simpler to implement than other proper ROC models. Expand
Evaluating probabilistic forecasts with Bayesian signal detection models.
This work proposes the use of signal detection theory (SDT) to evaluate the performance of both probabilistic forecasting systems and individual forecasters and shows how this approach allows ROC curves and AUCs to be applied to individuals within a group of forecasters, estimated as a function of time, and extended to measure differences in forecastability across different domains. Expand
Semiparametric Receiver Operating Characteristic Analysis to Evaluate Biomarkers for Disease
The receiver operating characteristic (ROC) curve is a popular method for characterizing the accuracy of diagnostic tests when test results are not binary. Various methodologies for estimating andExpand
Parameter Estimation for the Beta Distribution
PARAMETER ESTIMATION FOR THE BETA DISTRIBUTION Claire B. Owen Department of Statistics Master of Science The beta distribution is useful in modeling continuous random variables that lie between 0 andExpand
An introduction to ROC analysis
The purpose of this article is to serve as an introduction to ROC graphs and as a guide for using them in research. Expand