Corpus ID: 218684900

Fast cross-validation for multi-penalty ridge regression

  title={Fast cross-validation for multi-penalty ridge regression},
  author={M. V. D. Wiel and M. V. Nee and Armin Rauschenberger},
  journal={arXiv: Methodology},
Prediction based on multiple high-dimensional data types needs to account for the potentially strong differences in predictive signal. Ridge regression is a simple, yet versatile and interpretable model for high-dimensional data that has challenged the predictive performance of many more complex models and learners, in particular in dense settings. Moreover, it allows using a specific penalty per data type to account for differences between those. Then, the largest challenge for multi-penalty… Expand

Figures and Tables from this paper

Fast marginal likelihood estimation of penalties for group-adaptive elastic net
Nowadays, clinical research routinely uses omics data, such as gene expression, for predicting clinical outcomes or selecting markers. Additionally, so-called co-data are often available, providingExpand
Group-regularized ridge regression via empirical Bayes noise level cross-validation.
Features in predictive models are not exchangeable, yet common supervised models treat them as such. Here we study ridge regression when the analyst can partition the features into $K$ groups basedExpand


Efficient approximate k-fold and leave-one-out cross-validation for ridge regression.
  • R. Meijer, J. Goeman
  • Mathematics, Medicine
  • Biometrical journal. Biometrische Zeitschrift
  • 2013
An approximation method is discussed that is much faster and can be used in generalized linear models and Cox' proportional hazards model with a ridge penalty term and is most accurate when approximating leave-one-out cross-validation results for large data sets. Expand
Adaptive penalization in high-dimensional regression and classification with external covariates using variational Bayes
This work presents a method that differentially penalizes feature groups defined by the covariates and adapts the relative strength of penalization to the information content of each group, and extends the range of applications of penalized regression, improves model interpretability and can improve prediction performance. Expand
Better prediction by use of co-data: adaptive group-regularized ridge regression.
It is shown that the group-specific penalties may lead to a larger distinction between 'near-zero' and relatively large regression parameters, which facilitates post hoc variable selection and improves the predictive performances of ordinary logistic ridge regression and the group lasso. Expand
High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification
We provide a unified analysis of the predictive risk of ridge regression and regularized discriminant analysis in a dense random effects model. We work in a high-dimensional asymptotic regime whereExpand
Cross-study validation for the assessment of prediction algorithms
This work develops and implements a systematic approach to ‘cross-study validation’, to replace or supplement conventional cross-validation when evaluating high-dimensional prediction models in independent datasets, and suggests that standard cross- validation produces inflated discrimination accuracy for all algorithms considered, when compared to cross- study validation. Expand
Scalable Bayesian Regression in High Dimensions With Multiple Data Sources
Abstract Applications of high-dimensional regression often involve multiple sources or types of covariates. We propose methodology for this setting, emphasizing the “wide data” regime with largeExpand
Estimation of variance components, heritability and the ridge penalty in high-dimensional generalized linear models
For high-dimensional linear regression models, we review and compare several estimators of variances $\tau^2$ and $\sigma^2$ of the random slopes and errors, respectively. These variances relateExpand
Consistent High-Dimensional Bayesian Variable Selection via Penalized Credible Regions
  • H. Bondell, B. Reich
  • Computer Science, Medicine
  • Journal of the American Statistical Association
  • 2012
This work proposes a conjugate prior only on the full model parameters and use sparse solutions within posterior credible regions to perform selection, and shows that these sparse solutions can be computed via existing algorithms. Expand
Adaptive group-regularized logistic elastic net regression.
A group-regularized (logistic) elastic net regression method, where each penalty parameter corresponds to a group of features based on the external information, which shows that, if the partitioning of the features is informative, classification performance and feature selection are indeed enhanced. Expand
IPF-LASSO: Integrative L 1-Penalized Regression with Penalty Factors for Prediction Based on Multi-Omics Data
This paper proposes a simple penalized regression method, called IPF-LASSO (Integrative LASSO with Penalty Factors), and is implemented in the R package ipflasso and illustrated through applications to two real-life cancer datasets. Expand