Statistical comparison of classifiers through Bayesian hierarchical modelling

@article{Corani2017StatisticalCO,
  title={Statistical comparison of classifiers through Bayesian hierarchical modelling},
  author={Giorgio Corani and Alessio Benavoli and Janez Demsar and Francesca Mangili and Marco Zaffalon},
  journal={Machine Learning},
  year={2017},
  volume={106},
  pages={1817-1837}
}
  • Giorgio Corani, Alessio Benavoli, +2 authors Marco Zaffalon
  • Published 2017
  • Computer Science, Mathematics
  • Machine Learning
  • Usually one compares the accuracy of two competing classifiers using null hypothesis significance tests. Yet such tests suffer from important shortcomings, which can be overcome by switching to Bayesian hypothesis testing. We propose a Bayesian hierarchical model that jointly analyzes the cross-validation results obtained by two classifiers on multiple data sets. The model estimates more accurately the difference between classifiers on the individual data sets than the traditional approach of… CONTINUE READING

    Figures, Tables, and Topics from this paper.

    Explore key concepts

    Links to highly relevant papers for key concepts in this paper:

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 21 CITATIONS

    Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis

    VIEW 3 EXCERPTS
    CITES BACKGROUND & METHODS

    Violation of Independence of Irrelevant Alternatives in Friedman's test

    VIEW 1 EXCERPT
    CITES BACKGROUND

    Evaluating Predictive Models of Student Success: Closing the Methodological Gap

    VIEW 13 EXCERPTS
    CITES BACKGROUND & METHODS
    HIGHLY INFLUENCED

    Bayesian Evaluation and Comparison of Ontology Alignment Systems

    VIEW 2 EXCERPTS
    CITES METHODS & BACKGROUND

    Scalable Gaussian process-based transfer surrogates for hyperparameter optimization

    VIEW 2 EXCERPTS
    CITES BACKGROUND & METHODS

    On the Jeffreys-Lindley Paradox and the Looming Reproducibility Crisis in Machine Learning

    VIEW 1 EXCERPT
    CITES BACKGROUND

    FILTER CITATIONS BY YEAR

    2017
    2020

    CITATION STATISTICS

    • 2 Highly Influenced Citations

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 29 REFERENCES