• Corpus ID: 195767546

# Efficient Regularized Piecewise-Linear Regression Trees

@article{Lefakis2019EfficientRP,
title={Efficient Regularized Piecewise-Linear Regression Trees},
author={Leonidas Lefakis and Oleksandr Zadorozhnyi and Gilles Blanchard},
journal={ArXiv},
year={2019},
volume={abs/1907.00275}
}
• Published 29 June 2019
• Computer Science, Mathematics
• ArXiv
We present a detailed analysis of the class of regression decision tree algorithms which employ a regulized piecewise-linear node-splitting criterion and have regularized linear models at the leaves. From a theoretic standpoint, based on Rademacher complexity framework, we present new high-probability upper bounds for the generalization error for the proposed classes of regularized regression decision tree algorithms, including LASSO-type, and $\ell_{2}$ regularization for linear models at the…
2 Citations

## References

SHOWING 1-10 OF 29 REFERENCES

### Scalable look-ahead linear regression trees

• Computer Science
KDD '07
• 2007
The motivation behind Look-ahead Linear Regression Trees (LLRT) is that out of all the methods proposed to date, there has been no scalable approach to exhaustively evaluate all possible models in the leaf nodes in order to obtain an optimal split.

### Efficient Non-greedy Optimization of Decision Trees

• Computer Science
NIPS
• 2015
It is shown that the problem of finding optimal linear-combination splits for decision trees is related to structured prediction with latent variables, and a convex-concave upper bound on the tree's empirical loss is formed, and the use of stochastic gradient descent for optimization enables effective training with large datasets.

### Error Bounds for Piecewise Smooth and Switching Regression

• Fabien Lauer
• Mathematics, Computer Science
IEEE Transactions on Neural Networks and Learning Systems
• 2020
This paper derives generalization error bounds in piecewise smooth (PWS) regression and switching regression by following the approach based on Rademacher complexities, and shows how to recover a radical dependence.

### Risk Bounds for CART Regression Trees

• Computer Science, Mathematics
• 2003
The aim of the Classification And Regression Trees (CART) proposed by Breiman, Friedman, Olshen and Stone [1] in 1984 is to construct some efficient algorithm which gives a piecewise constant

### REGRESSION TREES WITH UNBIASED VARIABLE SELECTION AND INTERACTION DETECTION

The proposed algorithm, GUIDE, is specifically designed to eliminate variable selection bias, a problem that can undermine the reliability of inferences from a tree structure and allows fast computation speed, natural ex- tension to data sets with categorical variables, and direct detection of local two- variable interactions.

### SECRET: a scalable linear regression tree algorithm

• Computer Science
KDD
• 2002
A novel regression tree construction algorithm (SECRET) that produces trees of high quality and scales to very large datasets that has accuracy comparable to other linear regression tree algorithms but takes orders of magnitude less computation time for large datasets.

### Piecewise polynomial regression trees

• Mathematics, Computer Science
• 1994
The advantages of the proposed estimate are that it possesses a smooth analytic form, is as many times differentiable as the family of weight functions are, and has a decision tree representation.

### Random Composite Forests

• Computer Science
AAAI
• 2016
A new algorithm, RANDOMCOMPOSITEFORESTS (RCF), that is based on forming an ensemble of random composite trees that yields significant performance improvements over both Random Forests and a variant of RCF in several tasks is devised.

### Rademacher penalties and structural risk minimization

This work suggests a penalty function to be used in various problems of structural risk minimization, based on the sup-norm of the so-called Rademacher process indexed by the underlying class of functions (sets), and obtains oracle inequalities for the theoretical risk of estimators, obtained by structural minimization of the empirical risk withRademacher penalties.

### BART: Bayesian Additive Regression Trees

• Computer Science
• 2010
We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian