Automatic Gradient Boosting

@article{Thomas2018AutomaticGB,
  title={Automatic Gradient Boosting},
  author={Janek Thomas and Stefan Coors and B. Bischl},
  journal={ArXiv},
  year={2018},
  volume={abs/1807.03873}
}
Automatic machine learning performs predictive modeling with high performing machine learning tools without human interference. This is achieved by making machine learning applications parameter-free, i.e. only a dataset is provided while the complete model selection and model building process is handled internally through (often meta) optimization. Projects like Auto-WEKA and auto-sklearn aim to solve the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem resulting in… Expand
Automatic Hyper-Parameter Tuning for Gradient Boosting Machine
TLDR
This paper has implemented the GA variation called hyper-genetic to tune the hyperparameter of GBE and explored the best setting possible of the genetic parameters and compared the results from Grid-search, Bayesian optimization, and random approach over four sets of data. Expand
Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization
TLDR
An overview of theoretical and practical results for popular machine learning algorithms, accompanied by an experimental analysis of 30 hyperparameters from six relevant machineLearning algorithms is presented. Expand
Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features
TLDR
Throughout this study, regularized versions of target encoding consistently provided the best results, and traditional encodings that make unreasonable assumptions to map levels to integers or to reduce the number of levels before creating binary indicator variables were not as effective. Expand
Multi-Objective Automatic Machine Learning with AutoxgboostMC
TLDR
This work proposes to optimize additional criteria defined by the user directly to guide the search towards an optimal machine learning pipeline, and provides a simple multi-criteria AutoML system and showcase an exemplary application. Expand
Machine Learning Automation Toolbox ( MLAUT )
In this paper we present MLAUT (Machine Learning AUtomation Toolbox) for 1 the python data science ecosystem. MLAUT automates large-scale evaluation 2 and benchmarking of machine learning algorithmsExpand
Machine Learning Automation Toolbox (MLaut)
TLDR
It is found that deep neural networks do not perform well on basic supervised learning, i.e., outside the more specialized, image-, audio-, or text-based tasks. Expand
H2O AutoML: Scalable Automatic Machine Learning
H2O is an open source, distributed machine learning platform designed to scale to very large datasets, with APIs in R, Python, Java and Scala. We present H2O AutoML, a highly scalable,Expand
AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
We introduce AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models on an unprocessed tabular dataset such as aExpand
AutonoML: Towards an Integrated Framework for Autonomous Machine Learning
TLDR
This review seeks to motivate a more expansive perspective on what constitutes an automated/autonomous ML system, alongside consideration of how best to consolidate those elements, and develops a conceptual framework to illustrate one possible way of fusing high-level mechanisms into an autonomous ML system. Expand
An Open Source AutoML Benchmark
TLDR
An open, ongoing, and extensible benchmark framework which follows best practices and avoids common mistakes is introduced which is used to conduct a thorough comparison of 4 AutoML systems across 39 datasets and analyze the results. Expand
...
1
2
...

References

SHOWING 1-10 OF 22 REFERENCES
Efficient and Robust Automated Machine Learning
TLDR
This work introduces a robust new AutoML system based on scikit-learn, which improves on existing AutoML methods by automatically taking into account past performance on similar datasets, and by constructing ensembles from the models evaluated during the optimization. Expand
Practical Bayesian Optimization of Machine Learning Algorithms
TLDR
This work describes new algorithms that take into account the variable cost of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation and shows that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms. Expand
Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms
TLDR
This work considers the problem of simultaneously selecting a learning algorithm and setting its hyperparameters, going beyond previous work that attacks these issues separately and shows classification performance often much better than using standard selection and hyperparameter optimization methods. Expand
Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning
TLDR
This paper proposes a new kernel-based stochastic gradient descent algorithm that performs model selection while training, with no parameters to tune, nor any form of cross-validation, to estimate over time the right regularization in a data-dependent way. Expand
DART: Dropouts meet Multiple Additive Regression Trees
TLDR
A novel way of employing dropouts in MART is proposed, resulting in the DART algorithm, which outperforms MART in each of the tasks, with a significant margin, and overcomes the issue of over-specialization to a considerable extent. Expand
mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions
We present mlrMBO, a flexible and comprehensive R toolbox for model-based optimization (MBO), also known as Bayesian optimization, which addresses the problem of expensive black-box optimization byExpand
Hyperparameters and tuning strategies for random forest
TLDR
A literature review on the parameters' influence on the prediction performance and on variable importance measures is provided, and the application of one of the most established tuning strategies, model‐based optimization (MBO), is demonstrated. Expand
LightGBM: A Highly Efficient Gradient Boosting Decision Tree
TLDR
It is proved that, since the data instances with larger gradients play a more important role in the computation of information gain, GOSS can obtain quite accurate estimation of the information gain with a much smaller data size, and is called LightGBM. Expand
Greedy function approximation: A gradient boosting machine.
Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansionsExpand
mlr: Machine Learning in R
TLDR
The MLR package provides a generic, object-oriented, and extensible framework for classification, regression, survival analysis and clustering for the R language and includes meta-algorithms and model selection techniques to improve and extend the functionality of basic learners with, e.g., hyperparameter tuning, feature selection, and ensemble construction. Expand
...
1
2
3
...