Exploratory and Inferential Analysis of Benchmark Experiments

  title={Exploratory and Inferential Analysis of Benchmark Experiments},
  author={Manuel J. A. Eugster and Torsten Hothorn and Friedrich Leisch},
Benchmark experiments produce data in a very specific format. The observations are drawn from the performance distributions of the candidate algorithms on resampled data sets. In this paper we introduce a comprehensive toolbox of exploratory and inferential analysis methods for benchmark experiments based on one or more data sets. We present new visualization techniques, show how formal non-parametric and parametric test procedures can be used to evaluate the results, and, finally, how to sum… CONTINUE READING


Publications citing this paper.
Showing 1-10 of 19 extracted citations

Towards Bridging the Gap between Machine Learning Researchers and Practitioners

2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity) • 2015
View 1 Excerpt


Publications referenced by this paper.
Showing 1-10 of 18 references

Statistical Comparisons of Classifiers over Multiple Data Sets

Journal of Machine Learning Research • 2006
View 5 Excerpts
Highly Influenced

URL http://www.ics.uci.edu/ mlearn/MLRepository.html

A. Asuncion, D. J. Newman
UCI machine learning repository, • 2007
View 3 Excerpts
Highly Influenced

Mixed-Effects Models in S and S-PLUS

José C. Pinheiro, Douglas M. Bates
View 3 Excerpts
Highly Influenced

Simultaneous inference in general parametric models.

Biometrical journal. Biometrische Zeitschrift • 2008
View 1 Excerpt

Torsten Hothorn , Friedrich Leisch , Achim Zeileis , and Kurt Hornik . The design and analysis of benchmark experiments

Kurt Hornik Torsten Hothorn, Mark A. van de Wiel
Journal of Computational and Graphical Statistics • 2007

, and Achim Zeileis . A Lego system for conditional inference

Torsten Hothorn, Frank Bretz, Peter Westfall
The American Statistician • 2006
View 1 Excerpt

Similar Papers

Loading similar papers…