# On the non‐negative garrotte estimator

@article{Yuan2007OnTN, title={On the non‐negative garrotte estimator}, author={Ming Yuan and Yi Lin}, journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)}, year={2007}, volume={69} }

Summary. We study the non‐negative garrotte estimator from three different aspects: consistency, computation and flexibility. We argue that the non‐negative garrotte is a general procedure that can be used in combination with estimators other than the original least squares estimator as in its original form. In particular, we consider using the lasso, the elastic net and ridge regression along with ordinary least squares as the initial estimate in the non‐negative garrotte. We prove that the…

## 240 Citations

Variable selection in additive models by non-negative garrote

- Computer Science
- 2006

Breiman’s non-negative garrote method is adapted to perform variable selection in non-parametric additive models and provides accurate predictions and is effective at identifying the variables generating the model.

Consistency and robustness properties of the S-nonnegative garrote estimator

- Mathematics
- 2017

ABSTRACT This paper concerns a robust variable selection method in multiple linear regression: the robust S-nonnegative garrote variable selection method. In this paper the consistency of the method,…

The Adaptive Gril Estimator with a Diverging Number of Parameters

- Computer ScienceArXiv
- 2013

The grouped selection property for AdaCnet method (one type of AdaGril) in the equal correlation case is highlighted and it is shown that AdaGrils estimator achieves a Sparsity Inequality, i.e., a bound in terms of the number of non-zero components of the “true” regression coefficient.

Balancing stability and bias reduction in variable selection with the Mnet estimator

- Business
- 2013

Summary. We propose a new penalized approach for variable selection using a combination of minimax concave and ridge penalties. The proposed method is designed to deal with p n problems with highly…

Uniformly valid confidence sets based on the Lasso

- Mathematics, Computer Science
- 2018

In a linear regression model of fixed dimension, in finite samples with Gaussian errors and asymptotically in the case where the Lasso estimator is tuned to perform conservative model selection, exact formulas are derived for computing the minimal coverage probability over the entire parameter space for a large class of shapes for the confidence sets, thus enabling the construction of valid confidence regions based on the Lello estimator in these settings.

An ordinary differential equation-based solution path algorithm

- Computer Science, MathematicsJournal of nonparametric statistics
- 2011

This work proposes an extension of the LAR for generalised linear models and the quasi-likelihood model by showing that the corresponding solution path is piecewise given by solutions of ordinary differential equation (ODE) systems.

Estimation Consistency of the Group Lasso and its Applications

- Computer Science, EconomicsAISTATS
- 2009

The main theorem shows that the group Lasso achieves estimation consistency under a mild condition and an asymptotic upper bound on the number of selected variables can be obtained.

Some Notes on the Nonnegative Garrote

- MathematicsTechnometrics
- 2010

The main result is that, compared with other penalized least-squares methods, the NG has a natural selection of penalty function according to an estimator of prediction risk, indicating that to select tuning parameters, it may be unnecessary to optimize a model selection criterion repeatedly.

Model-Consistent Sparse Estimation through the Bootstrap

- Computer Science, MathematicsArXiv
- 2009

This paper first presents a detailed asymptotic analysis of model consistency of the Lasso in low-dimensional settings, and shows that if the authors run theLasso for several bootstrapped replications of a given sample, then intersecting the supports ofThe Lasso bootstrap estimates leads to consistent model selection.

Resampling-Based Variable Selection with Lasso for p >> n and Partially Linear Models

- Computer Science2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA)
- 2015

It is demonstrated theoretically that false positives are likely to be selected by the Lasso method due to a small proportion of samples, which happen to explain some variation in the response variable, and a novel consistent variable selection algorithm is proposed based on this property.

## References

SHOWING 1-10 OF 22 REFERENCES

On the LASSO and its dual

- Computer Science, Mathematics
- 2000

Consideration of the primal and dual problems together leads to important new insights into the characteristics of the LASSO estimator and to an improved method for estimating its covariance matrix.

Better subset regression using the nonnegative garrote

- Mathematics
- 1995

A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeroes coefficients. In tests on real and simulated data, it produces lower prediction…

Model selection and estimation in regression with grouped variables

- Mathematics
- 2006

Summary. We consider the problem of selecting grouped variables (factors) for accurate prediction in regression. Such a problem arises naturally in many practical situations with the multifactor…

Regression Shrinkage and Selection via the Lasso

- Computer Science
- 1996

A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.

The Adaptive Lasso and Its Oracle Properties

- Computer Science
- 2006

A new version of the lasso is proposed, called the adaptive lasso, where adaptive weights are used for penalizing different coefficients in the ℓ1 penalty, and the nonnegative garotte is shown to be consistent for variable selection.

Regularization and variable selection via the elastic net

- Computer Science
- 2005

It is shown that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation, and an algorithm called LARS‐EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lamba.

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

- Mathematics, Computer Science
- 2001

In this article, penalized likelihood approaches are proposed to handle variable selection problems, and it is shown that the newly proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well if the correct submodel were known.

High-dimensional graphs and variable selection with the Lasso

- Computer Science
- 2006

It is shown that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs and is hence equivalent to variable selection for Gaussian linear models.

Heuristics of instability and stabilization in model selection

- Mathematics
- 1996

In model selection, usually a best predictor is chosen from a collection {μ(.,s)} of predictors where μ(.,s) is the minimum least-squares predictor in a collection U s of predictors. Here s is a…

Calibration and empirical Bayes variable selection

- Mathematics
- 2000

For the problem of variable selection for the normal linear model, selection criteria such as AIC, C p , BIC and RIC have fixed dimensionality penalties. Such criteria are shown to correspond to…