Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference

  title={Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference},
  author={Max Goplerud},
  journal={arXiv: Methodology},
  • Max Goplerud
  • Published 2020
  • Computer Science, Mathematics
  • arXiv: Methodology
Estimating non-linear hierarchical models can be computationally burdensome in the presence of large datasets and many non-nested random effects. Popular inferential techniques may take hours to fit even relatively straightforward models. This paper provides two contributions to scalable and accurate inference. First, I propose a new mean-field algorithm for estimating logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose "marginally augmented… Expand
Scalable computation for Bayesian hierarchical models
The article is about algorithms for learning Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and the number of parameters in theExpand


Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations
The effects of different parametrizations on the convergence of Bayesian computational algorithms for hierarchical models are well explored. Techniques such as centering, noncentering and partialExpand
Fast and Accurate Binary Response Mixed Model Analysis via Expectation Propagation
Numerical studies reveal the availability of fast, highly accurate and scalable methodology for binary mixed model analysis and show that fast and accurate quadrature-free inference can be realized for the probit link case with multivariate random effects and higher levels of nesting. Expand
Semi-Implicit Variational Inference
With a substantially expanded variational family and a novel optimization algorithm, SIVI is shown to closely match the accuracy of MCMC in inferring the posterior in a variety of Bayesian inference tasks. Expand
Weakly Informative Prior for Point Estimation of Covariance Matrices in Hierarchical Models
When fitting hierarchical regression models, maximum likelihood (ML) estimation has computational (and, for some users, philosophical) advantages compared to full Bayesian inference, but when theExpand
A Variational Maximization–Maximization Algorithm for Generalized Linear Mixed Models with Crossed Random Effects
Numerical studies show that under the small sample size conditions that are considered the proposed variational maximization–maximization algorithm outperforms the Laplace approximation. Expand
Streamlined Variational Inference for Linear Mixed Models with Crossed Random Effects
We derive streamlined mean field variational Bayes algorithms for fitting linear mixed models with crossed random effects. In the most general situation, where the dimensions of the crossed groupsExpand
Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Pólya–Gamma distributions, which are constructedExpand
Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
An efficient computation of LOO is introduced using Pareto-smoothed importance sampling (PSIS), a new procedure for regularizing importance weights, and it is demonstrated that PSIS-LOO is more robust in the finite case with weak priors or influential observations. Expand
Gaussian Variational Approximate Inference for Generalized Linear Mixed Models
Variational approximation methods have become a mainstay of contemporary machine learning methodology, but currently have little presence in statistics. We devise an effective variationalExpand
The one-step-late PXEM algorithm
The one-step-late EM algorithm is adapted to PXEM to establish a fast closed form algorithm that improves on the one- Step-Late EM algorithm by insuring monotone convergence, and is used to fit a probit regression model and a variety of dynamic linear models. Expand