• Corpus ID: 235359246

Spike-and-Slab Group Lasso for Consistent Estimation and Variable Selection in Non-Gaussian Generalized Additive Models.

@article{Bai2020SpikeandSlabGL,
  title={Spike-and-Slab Group Lasso for Consistent Estimation and Variable Selection in Non-Gaussian Generalized Additive Models.},
  author={Ray Bai},
  journal={arXiv: Methodology},
  year={2020}
}
  • Ray Bai
  • Published 14 July 2020
  • Computer Science
  • arXiv: Methodology
We study estimation and variable selection in non-Gaussian Bayesian generalized additive models (GAMs) under a spike-and-slab prior for grouped variables. Our framework subsumes GAMs for logistic regression, Poisson regression, negative binomial regression, and gamma regression, and encompasses both canonical and non-canonical link functions. Under mild conditions, we establish posterior contraction rates and model selection consistency when $p \gg n$. For computation, we propose an EM… 
View PDF on arXiv
5 Citations
Methods Citations
2
Results Citations
1

Figures from this paper

5 Citations

Citation Type

Citation Type

A scalable and flexible Cox proportional hazards model for high-dimensional survival prediction and functional selection
TLDR
An additive Cox PH model is developed that imposes a novel spike-and-slab LASSO prior that motivates the bi-level functional selection on additive functions and compares the predictive and computational performance against state-of-the-art models in simulation studies and metabolomics data analysis.
Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes
TLDR
This work develops a fast and scalable variational inference algorithm for the spike and slab GP that is tractable with arbitrary differentiable kernels and improves the algorithm’s ability to adapt to the sparsity of relevant variables by Bayesian model averaging over hyperparameters.
Spike-and-Slab Generalized Additive Models and Scalable Algorithms for High-Dimensional Data
TLDR
Bayesian hierarchical generalized additive models are proposed that consider the smoothing penalty for proper shrinkage of curve interpolation and separation of smoothing function linear and nonlinear spaces, and two scalable and deterministic algorithms are developed for different utilities.
Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis
TLDR
A Bayesian hierarchical generalized additive models is proposed that considers the smoothing penalty for proper shrinkage of curve interpolation via reparameterization and a novel two-part spike-and-slab LASSO prior for smoothing functions is developed to address the sparsity of signals.
Posterior Concentration Rates for Bayesian Penalized Splines
TLDR
This work studies posterior concentration rates for Bayesian penalized splines in a Gaussian nonparametric regression model and shows that posterior concentration at near optimal rate can be achieved if the hyperprior on the smoothing variance strikes a fine balance between oversmoothing and undersmoothing.

References

SHOWING 1-10 OF 27 REFERENCES
Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models
Summary.  Recent work by Reiss and Ogden provides a theoretical basis for sometimes preferring restricted maximum likelihood (REML) to generalized cross‐validation (GCV) for smoothing parameter…
Posterior contraction in sparse generalized linear models
TLDR
This work studies posterior contraction rates in sparse high-dimensional generalized linear models using priors incorporating sparsity, and shows that Bayesian methods achieve convergence properties analogous to lasso-type procedures.
The Spike-and-Slab LASSO
TLDR
It is shown on simulated data that the fully Bayes penalty mimics oracle performance, providing a viable alternative to cross-validation and developing theory for the separable and nonseparable variants of the penalty.
EMVS: The EM Approach to Bayesian Variable Selection
TLDR
EMVS is proposed, a deterministic alternative to stochastic search based on an EM algorithm which exploits a conjugate mixture prior formulation to quickly find posterior modes in high-dimensional linear regression contexts.
Bayesian variable selection for high dimensional generalized linear models : Convergence rates of the fitted densities
TLDR
It is shown that it is possible to use Bayesian variable selection to reduce overfitting caused by the curse of dimensionality K » n, if most of the xj's have very small effects on the response y, and a suitable prior can be used to choose a few out of the many x j 's to model y.
Bayesian group selection in logistic regression with application to MRI data analysis
TLDR
A hierarchical group spike and slab prior for logistic regression models in high‐dimensional settings is considered and strong group selection consistency of the induced posterior is established, the first theoretical result in the Bayesian literature.
Spike and-Slab Group LASSOs for Grouped Regression and Sparse Generalized Additive Models
Empirical Bayes oracle uncertainty quantification for regression
TLDR
It is shown that an empirical Bayes posterior contracts at the optimal rate at all parameters and leads to uniform size-optimal credible balls with guaranteed coverage under an “excessive bias restriction” condition.
CONVERGENCE RATES OF POSTERIOR DISTRIBUTIONS FOR NONIID OBSERVATIONS By
We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general…
Variable selection consistency of Gaussian process regression
TLDR
It is shown here that variable consistency may indeed be achieved withBayesian nonparametric regression under a rescaled Gaussian process prior at least when the true regression function has finite smoothness to induce a polynomially larger penalty on inclusion of false positive predictors.
...
...