Share This Author
Proceedings of the 28th International Conference on Machine Learning
An experimental comparison of performance measures for classification
Learning Decision Trees Using the Area Under the ROC Curve
This paper shows how a single decision tree can represent a set of classifiers by choosing different labellings of its leaves, or equivalently, an ordering on the leaves, and proposes a novel splitting criterion which chooses the split with the highest local AUC.
Volume under the ROC Surface for Multi-class Problems
This paper presents the real extension to the Area Under the AUC Curve in the form of the Volume Under the ROC Surface (VUS), showing how to compute the polytope that corresponds to the absence of classifiers, to the best classifier and to whatever set of classifier.
Quantification via Probability Estimators
- Antonio Bella, C. Ferri, J. Hernández-Orallo, M. J. Ramírez-Quintana
- Computer ScienceIEEE International Conference on Data Mining
- 13 December 2010
A method based on averaging the probability estimations of a classifier with a very simple scaling that does perform reasonably well is presented, showing that probability estimators for quantification capture a richer view of the problem than methods based on a threshold.
A Coherent Interpretation of AUC as a Measure of Aggregated Classification Performance
Should one wish to consider only optimal thresholds, it is demonstrated that a simple and more intuitive alternative to Hand's H measure is already available in the form of the area under the cost curve, both uniform and hence model-independent.
Measuring universal intelligence: Towards an anytime intelligence test
Evaluation in artificial intelligence: from task-oriented to ability-oriented measurement
- J. Hernández-Orallo
- Computer ScienceArtificial Intelligence Review
- 1 October 2017
This paper critically assess the different ways AI systems are evaluated, and the role of components and techniques in these systems, and identifies three kinds of evaluation: human discrimination, problem benchmarks and peer confrontation.
A unified view of performance metrics: translating threshold choice into expected classification loss
This analysis provides a comprehensive view of performance metrics as well as a systematic approach to loss minimisation which can be summarised as follows: given a model, apply the threshold choice methods that correspond with the available information about the operating condition, and compare their expected losses.
Inductive programming meets the real world
- Sumit Gulwani, J. Hernández-Orallo, E. Kitzelmann, S. Muggleton, Ute Schmid, B. Zorn
- Computer ScienceCommun. ACM
- 23 October 2015
Inductive programming can liberate users from performing tedious and repetitive tasks by enabling them to focus on solving real-time problems.