Regularization and variable selection via the elastic net

@article{Zou2005RegularizationAV,
  title={Regularization and variable selection via the elastic net},
  author={Hui Zou and Trevor J. Hastie},
  journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
  year={2005},
  volume={67}
}
  • H. Zou, T. Hastie
  • Published 1 April 2005
  • Computer Science
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology)
Summary.  We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n… 
The Bayesian elastic net
TLDR
A Bayesian method to solve the elastic net model using a Gibbs sampler and shows that the Bayesian elastic net behaves comparably in prediction accuracy but performs better in variable selection.
Elastic Net Procedure for Partially Linear Models
TLDR
This paper proposes Elastic Net procedure for partially linear models and proves the group effect of its estimate and shows that the strongly correlated variables problem can be better handled by the Elastic net procedure than Lasso, ALasso and Ridge.
ON THE ADAPTIVE ELASTIC-NET WITH A DIVERGING NUMBER OF PARAMETERS.
TLDR
This paper proposes the adaptive Elastic-Net that combines the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage and establishes the oracle property of the adaptive elastic-Net under weak regularity conditions.
Penalized regression with correlation-based penalty
TLDR
A new regularization method for regression models which contains a penalty term which explicitly links strength of penalization to the correlation between predictors, and which compares well to competing regularization techniques.
A group adaptive elastic-net approach for variable selection in high-dimensional linear regression
In practice, predictors possess grouping structures spontaneously. Incorporation of such useful information can improve statistical modeling and inference. In addition, the high-dimensionality often
Exploiting Covariate Similarity in Sparse Regression via the Pairwise Elastic Net
TLDR
A new approach to regression regularization called the Pairwise Elastic Net, which simultaneously performs automatic variable selection and continuous shrinkage and encourages the grouping of strongly correlated predictors based on a pairwise similarity measure is proposed.
Regression and Variable Selection via A Layered Elastic Net
One approach in modeling high dimensional data is to apply an elastic net (EN) regularization framework. EN has the good properties of least absolute shrinkage selection operator (LASSO), however, EN
On the grouped selection and model complexity of the adaptive elastic net
TLDR
This article focuses on the grouped selection property of adaptive elastic net along with its model selection complexity and sheds some light on the bias-variance tradeoff of different regularization methods including adaptive elasticNet.
Elastic Net Regularization Paths for All Generalized Linear Models
TLDR
The reach of the elastic net-regularized regression is extended to all generalized linear model families, Cox models with (start, stop] data and strata, and a simplified version of the relaxed lasso.
Adaptive elastic net-penalized quantile regression for variable selection
Abstract There has been much attention on the high-dimensional linear regression models, which means the number of observations is much less than that of covariates. Considering the fact that the
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 37 REFERENCES
Addendum: Regularization and variable selection via the elastic net
TLDR
The piecewise linearity of the lasso solution path was first proved by Osborne et al. (2000), who also described an efficient algorithm for calculating the complete lasso solutions path.
Regression Shrinkage and Selection via the Lasso
TLDR
A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Penalized Regressions: The Bridge versus the Lasso
TLDR
It is shown that the bridge regression performs well compared to the lasso and ridge regression, and is demonstrated through an analysis of a prostate cancer data.
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
TLDR
In this article, penalized likelihood approaches are proposed to handle variable selection problems, and it is shown that the newly proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well if the correct submodel were known.
Sparse Principal Component Analysis
TLDR
This work introduces a new method called sparse principal component analysis (SPCA) using the lasso (elastic net) to produce modified principal components with sparse loadings and shows that PCA can be formulated as a regression-type optimization problem.
A new approach to variable selection in least squares problems
TLDR
A compact descent method for solving the constrained problem for a particular value of κ is formulated, and a homotopy method, in which the constraint bound κ becomes the Homotopy parameter, is developed to completely describe the possible selection regimes.
Heuristics of instability and stabilization in model selection
In model selection, usually a best predictor is chosen from a collection {μ(.,s)} of predictors where μ(.,s) is the minimum least-squares predictor in a collection U s of predictors. Here s is a
Boosting as a Regularized Path to a Maximum Margin Classifier
TLDR
It is built on recent work by Efron et al. to show that boosting approximately (and in some cases exactly) minimizes its loss criterion with an l1 constraint on the coefficient vector, and shows that as the constraint is relaxed the solution converges (in the separable case) to an "l1-optimal" separating hyper-plane.
Regularized Discriminant Analysis
TLDR
Alternatives to the usual maximum likelihood estimates for the covariance matrices are proposed, characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk.
Statistical behavior and consistency of classification methods based on convex risk minimization
TLDR
This study sheds light on the good performance of some recently proposed linear classification methods including boosting and support vector machines and shows their limitations and suggests possible improvements.
...
1
2
3
4
...