Regression Shrinkage and Selection via the Lasso

@article{Tibshirani1996RegressionSA,
  title={Regression Shrinkage and Selection via the Lasso},
  author={Robert Tibshirani},
  journal={Journal of the royal statistical society series b-methodological},
  year={1996},
  volume={58},
  pages={267-288}
}
  • R. Tibshirani
  • Published 1996
  • Computer Science
  • Journal of the royal statistical society series b-methodological
SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable… 

A random model approach for the LASSO

TLDR
Two related simulation studies are presented that show that dispersion parameter estimation results in effect estimates that are competitive with other estimation methods (including other LASSO methods).

High-Dimensional LASSO-Based Computational Regression Models: Regularization, Shrinkage, and Selection

TLDR
The regularization terms responsible for inducing coefficient shrinkage and variable selection leading to improved performance metrics of these regression models are discussed, making these modern, computational regression models valuable tools for analyzing high-dimensional problems.

CG-Lasso Estimator for Multivariate Adaptive Regression Spline

TLDR
The Tikhonov penalty function is changed with the generalized Lasso penalty for solving the problem PRSS, taking an advantage for feature selection and is applied to the elegant framework of conic quadratic programming (CQP), and is called as CG-Lasso.

A proposal for variable selection in the Cox model

TLDR
Simulations indicate that the lasso can be more accurate than stepwise selection in this setting and give interpretable models in Cox's proportional hazards model.

LAD, LASSO and Related Strategies in Regression Models

In the context of linear regression models, it is well-known that the ordinary least squares estimator is very sensitive to outliers whereas the least absolute deviations (LAD) is an alternative

On the LASSO and its Dual

TLDR
Consideration of the primal and dual problems together leads to important new insights into the characteristics of the LASSO estimator and to an improved method for estimating its covariance matrix.

The Iterated Lasso for High-Dimensional Logistic Regression By

TLDR
This work considers an iterated Lasso approach for variable selection and estimation in sparse, high-dimensional logistic regression models and provides conditions under which this two-step approach possesses asymptotic oracle Selection and estimation properties.

Regression coefficient and autoregressive order shrinkage and selection via the lasso

TLDR
This work proposes a second lasso estimator which uses different tuning parameters for each coefficient and shows that this modified lasso can produce the estimator as efficiently as the oracle, and proposes an algorithm for tuning parameter estimates to obtain the modified lwheel estimator.

On ridge regression and least absolute shrinkage and selection operator

This thesis focuses on ridge regression (RR) and least absolute shrinkage and selection operator (lasso). Ridge properties are being investigated in great detail which include studying the bias, the

Mixed Lasso estimator for stochastic restricted regression models

TLDR
A Mixed Lasso (M-Lasso) estimator that incorporates stochastic linear restrictions to big data sets for selecting the true model and estimating parameters simultaneously and can provide reasonable and more precise estimates of model parameters that are in line with the economic theory.
...

References

SHOWING 1-10 OF 20 REFERENCES

A proposal for variable selection in the Cox model

TLDR
Simulations indicate that the lasso can be more accurate than stepwise selection in this setting and give interpretable models in Cox's proportional hazards model.

Variable selection via Gibbs sampling

Abstract A crucial problem in building a multiple regression model is the selection of predictors to include. The main thrust of this article is to propose and develop a procedure that uses

Submodel selection and evaluation in regression. The X-random case

Summary Often, in a regression situation with many variables, a sequence of submodels is generated containing fewer variables by using such methods as stepwise addition or deletion of variables, or

Linear Model Selection by Cross-validation

Abstract We consider the problem of selecting a model having the best predictive ability among a class of linear models. The popular leave-one-out cross-validation method, which is asymptotically

Model Selection Via Multifold Cross Validation

TLDR
Two notions of multi-fold cross validation (MCV and MCV*) criteria are considered and it turns out that MCV indeed reduces the chance of overfitting.

Bootstrap Methods: Another Look at the Jackknife

We discuss the following problem given a random sample X = (X 1, X 2,…, X n) from an unknown probability distribution F, estimate the sampling distribution of some prespecified random variable R(X,

Solving least squares problems

Since the lm function provides a lot of features it is rather complicated. So we are going to instead use the function lsfit as a model. It computes only the coefficient estimates and the residuals.

Ideal spatial adaptation by wavelet shrinkage

SUMMARY With ideal spatial adaptation, an oracle furnishes information about how best to adapt a spatially variable estimator, whether piecewise constant, piecewise polynomial, variable knot spline,

Wavelet Shrinkage: Asymptopia?

TLDR
A method for curve estimation based on n noisy data: translate the empirical wavelet coefficients towards the origin by an amount √(2 log n) /√n and draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves.

A Statistical View of Some Chemometrics Regression Tools

Chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis. In addition to borrowing many techniques from the statistics and engineering