Corpus ID: 26296192

OpenML Benchmarking Suites and the OpenML100

@article{Bischl2017OpenMLBS,
  title={OpenML Benchmarking Suites and the OpenML100},
  author={B. Bischl and Giuseppe Casalicchio and M. Feurer and F. Hutter and Michel Lang and R. Mantovani and J. Rijn and J. Vanschoren},
  journal={ArXiv},
  year={2017},
  volume={abs/1708.03731}
}
  • B. Bischl, Giuseppe Casalicchio, +5 authors J. Vanschoren
  • Published 2017
  • Mathematics, Computer Science
  • ArXiv
  • We advocate the use of curated, comprehensive benchmark suites of machine learning datasets, backed by standardized OpenML-based interfaces and complementary software toolkits written in Python, Java and R. Major distinguishing features of OpenML benchmark suites are (a) ease of use through standardized data formats, APIs, and existing client libraries; (b) machine-readable meta-information regarding the contents of the suite; and (c) online sharing of results, enabling large scale comparisons… CONTINUE READING
    33 Citations
    Benchmarking Automatic Machine Learning Frameworks
    • 25
    • PDF
    Benchmark and Survey of Automated Machine Learning Frameworks.
    • 27
    • PDF
    Decoding machine learning benchmarks
    Automatic Exploration of Machine Learning Experiments on OpenML
    • 5
    • PDF
    The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development
    • 10
    • PDF
    An ADMM Based Framework for AutoML Pipeline Configuration
    • 16
    • PDF
    An Empirical Study of Hyperparameter Importance Across Datasets
    • 5
    • PDF
    Meta learning for defaults: symbolic defaults
    • 6
    • PDF
    Two-stage Optimization for Machine Learning Workflow
    Hyperparameter Importance Across Datasets
    • 45
    • PDF

    References

    SHOWING 1-10 OF 15 REFERENCES
    PMLB: a large benchmark suite for machine learning evaluation and comparison
    • 119
    • PDF
    OpenML: An R package to connect to the machine learning platform OpenML
    • 25
    • PDF
    Sharing RapidMiner Workflows and Experiments with OpenML
    • 5
    • PDF
    Scikit-learn: Machine Learning in Python
    • 27,213
    • PDF
    OpenML: networked science in machine learning
    • 485
    • PDF
    Multilabel Classification with R Package mlr
    • 12
    • PDF
    The WEKA data mining software: an update
    • 18,382
    • PDF
    Generalizing from Case studies: A Case Study
    • 278
    On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration
    • 1,154
    • PDF
    KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework
    • 1,561
    • PDF