OpenML Benchmarking Suites and the OpenML100
@article{Bischl2017OpenMLBS, title={OpenML Benchmarking Suites and the OpenML100}, author={B. Bischl and Giuseppe Casalicchio and M. Feurer and F. Hutter and Michel Lang and R. Mantovani and J. Rijn and J. Vanschoren}, journal={ArXiv}, year={2017}, volume={abs/1708.03731} }
We advocate the use of curated, comprehensive benchmark suites of machine learning datasets, backed by standardized OpenML-based interfaces and complementary software toolkits written in Python, Java and R. Major distinguishing features of OpenML benchmark suites are (a) ease of use through standardized data formats, APIs, and existing client libraries; (b) machine-readable meta-information regarding the contents of the suite; and (c) online sharing of results, enabling large scale comparisons… CONTINUE READING
Supplemental Code
Github Repo
Via Papers with Code
Set of benchmarks comparing different Target Encoding options
Tables and Topics from this paper
Tables
Paper Mentions
33 Citations
Benchmark and Survey of Automated Machine Learning Frameworks.
- Computer Science, Mathematics
- 2019
- 37
- PDF
Automatic Exploration of Machine Learning Experiments on OpenML
- Mathematics, Computer Science
- ArXiv
- 2018
- 5
- PDF
The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development
- Computer Science, Mathematics
- SIGMOD Conference
- 2020
- 10
- PDF
An Empirical Study of Hyperparameter Importance Across Datasets
- Computer Science
- AutoML@PKDD/ECML
- 2017
- 6
- PDF
References
SHOWING 1-10 OF 15 REFERENCES
PMLB: a large benchmark suite for machine learning evaluation and comparison
- Computer Science, Medicine
- BioData Mining
- 2017
- 126
- PDF
OpenML: An R package to connect to the machine learning platform OpenML
- Computer Science, Mathematics
- Comput. Stat.
- 2019
- 26
- PDF
On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration
- Computer Science
- KDD '02
- 2002
- 1,163
- PDF
KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework
- Computer Science
- J. Multiple Valued Log. Soft Comput.
- 2011
- 1,599
- PDF