Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms

Abstract

Many different machine learning algorithms exist; taking into account each algorithm's hyperparameters, there is a staggeringly large number of possible alternatives overall. We consider the problem of simultaneously selecting a learning algorithm and setting its hyperparameters, going beyond previous work that attacks these issues separately. We show that this problem can be addressed by a fully automated approach, leveraging recent innovations in Bayesian optimization. Specifically, we consider a wide range of feature selection techniques (combining 3 search and 8 evaluator methods) and all classification approaches implemented in WEKA's standard distribution, spanning 2 ensemble methods, 10 meta-methods, 27 base classifiers, and hyperparameter settings for each classifier. On each of 21 popular datasets from the UCI repository, the KDD Cup 09, variants of the MNIST dataset and CIFAR-10, we show classification performance often much better than using standard selection and hyperparameter optimization methods. We hope that our approach will help non-expert users to more effectively identify machine learning algorithms and hyperparameter settings appropriate to their applications, and hence to achieve improved performance.

DOI: 10.1145/2487575.2487629
View Slides

Extracted Key Phrases

7 Figures and Tables

05010020132014201520162017
Citations per Year

250 Citations

Semantic Scholar estimates that this publication has 250 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Thornton2013AutoWEKACS, title={Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms}, author={Chris Thornton and Frank Hutter and Holger H. Hoos and Kevin Leyton-Brown}, booktitle={KDD}, year={2013} }