# Regularization and variable selection via the elastic net

@article{Zou2005RegularizationAV, title={Regularization and variable selection via the elastic net}, author={Hui Zou and Trevor J. Hastie}, journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)}, year={2005}, volume={67} }

Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n…

## 13,997 Citations

The Bayesian elastic net

- Computer Science
- 2010

A Bayesian method to solve the elastic net model using a Gibbs sampler and shows that the Bayesian elastic net behaves comparably in prediction accuracy but performs better in variable selection.

Elastic Net Procedure for Partially Linear Models

- Computer Science
- 2015

This paper proposes Elastic Net procedure for partially linear models and proves the group effect of its estimate and shows that the strongly correlated variables problem can be better handled by the Elastic net procedure than Lasso, ALasso and Ridge.

ON THE ADAPTIVE ELASTIC-NET WITH A DIVERGING NUMBER OF PARAMETERS.

- Computer ScienceAnnals of statistics
- 2009

This paper proposes the adaptive Elastic-Net that combines the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage and establishes the oracle property of the adaptive elastic-Net under weak regularity conditions.

Penalized regression with correlation-based penalty

- Computer ScienceStat. Comput.
- 2009

A new regularization method for regression models which contains a penalty term which explicitly links strength of penalization to the correlation between predictors, and which compares well to competing regularization techniques.

A group adaptive elastic-net approach for variable selection in high-dimensional linear regression

- Mathematics
- 2018

In practice, predictors possess grouping structures spontaneously. Incorporation of such useful information can improve statistical modeling and inference. In addition, the high-dimensionality often…

Exploiting Covariate Similarity in Sparse Regression via the Pairwise Elastic Net

- Computer ScienceAISTATS
- 2010

A new approach to regression regularization called the Pairwise Elastic Net, which simultaneously performs automatic variable selection and continuous shrinkage and encourages the grouping of strongly correlated predictors based on a pairwise similarity measure is proposed.

Regression and Variable Selection via A Layered Elastic Net

- Economics

One approach in modeling high dimensional data is to apply an elastic net (EN) regularization framework. EN has the good properties of least absolute shrinkage selection operator (LASSO), however, EN…

On the grouped selection and model complexity of the adaptive elastic net

- Computer ScienceStat. Comput.
- 2011

This article focuses on the grouped selection property of adaptive elastic net along with its model selection complexity and sheds some light on the bias-variance tradeoff of different regularization methods including adaptive elasticNet.

Elastic Net Regularization Paths for All Generalized Linear Models

- Computer Science
- 2021

The reach of the elastic net-regularized regression is extended to all generalized linear model families, Cox models with (start, stop] data and strata, and a simplified version of the relaxed lasso.

Adaptive elastic net-penalized quantile regression for variable selection

- MathematicsCommunications in Statistics - Theory and Methods
- 2019

Abstract There has been much attention on the high-dimensional linear regression models, which means the number of observations is much less than that of covariates. Considering the fact that the…

## References

SHOWING 1-10 OF 37 REFERENCES

Addendum: Regularization and variable selection via the elastic net

- Computer Science
- 2005

The piecewise linearity of the lasso solution path was first proved by Osborne et al. (2000), who also described an efficient algorithm for calculating the complete lasso solutions path.

Regression Shrinkage and Selection via the Lasso

- Computer Science
- 1996

A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.

Penalized Regressions: The Bridge versus the Lasso

- Computer Science
- 1998

It is shown that the bridge regression performs well compared to the lasso and ridge regression, and is demonstrated through an analysis of a prostate cancer data.

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

- Mathematics, Computer Science
- 2001

In this article, penalized likelihood approaches are proposed to handle variable selection problems, and it is shown that the newly proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well if the correct submodel were known.

Sparse Principal Component Analysis

- Computer Science
- 2006

This work introduces a new method called sparse principal component analysis (SPCA) using the lasso (elastic net) to produce modified principal components with sparse loadings and shows that PCA can be formulated as a regression-type optimization problem.

A new approach to variable selection in least squares problems

- Mathematics, Computer Science
- 2000

A compact descent method for solving the constrained problem for a particular value of κ is formulated, and a homotopy method, in which the constraint bound κ becomes the Homotopy parameter, is developed to completely describe the possible selection regimes.

Heuristics of instability and stabilization in model selection

- Mathematics
- 1996

In model selection, usually a best predictor is chosen from a collection {μ(.,s)} of predictors where μ(.,s) is the minimum least-squares predictor in a collection U s of predictors. Here s is a…

Boosting as a Regularized Path to a Maximum Margin Classifier

- Computer ScienceJ. Mach. Learn. Res.
- 2004

It is built on recent work by Efron et al. to show that boosting approximately (and in some cases exactly) minimizes its loss criterion with an l1 constraint on the coefficient vector, and shows that as the constraint is relaxed the solution converges (in the separable case) to an "l1-optimal" separating hyper-plane.

Regularized Discriminant Analysis

- Computer Science
- 1989

Alternatives to the usual maximum likelihood estimates for the covariance matrices are proposed, characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk.

Statistical behavior and consistency of classification methods based on convex risk minimization

- Computer Science
- 2003

This study sheds light on the good performance of some recently proposed linear classification methods including boosting and support vector machines and shows their limitations and suggests possible improvements.