Polynomial time guarantees for sampling based posterior inference in high-dimensional generalised linear models

@article{Altmeyer2022PolynomialTG,
  title={Polynomial time guarantees for sampling based posterior inference in high-dimensional generalised linear models},
  author={Randolf Altmeyer},
  journal={ArXiv},
  year={2022},
  volume={abs/2208.13296}
}
  • R. Altmeyer
  • Published 28 August 2022
  • Computer Science, Mathematics
  • ArXiv
The problem of computing posterior functionals in general high-dimensional statistical models with possibly non-log-concave likelihood functions is considered. Based on the proof strategy of [56], but using only local likelihood conditions and without relying on M-estimation theory, non-asymptotic statistical and computational guarantees are provided for gradient based MCMC algorithms. Given a suitable initialiser, these guarantees scale polynomially in key algorithmic quantities. The abstract… 
[PDF] Semantic Reader
2 Citations

2 Citations

On free energy barriers in Gaussian priors and failure of MCMC for high-dimensional unimodal distributions

We exhibit examples of high-dimensional unimodal posterior distributions arising in non-linear regression models with Gaussian process priors for which worst-case (‘cold start’) initialised MCMC…

On free energy barriers in Gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributions

We exhibit examples of high-dimensional unimodal posterior distributions arising in nonlinear regression models with Gaussian process priors for which Markov chain Monte Carlo (MCMC) methods can take…

77 References

On polynomial-time computation of high-dimensional posterior measures by Langevin-type algorithms

  • R. NicklSven Wang
  • Computer Science, Mathematics
    Journal of the European Mathematical Society
  • 2022
Non-asymptotic computational guarantees for Langevin-type MCMC algorithms which scale polynomially in key quantities such as the dimension of the model, the desired precision level, and the number of available statistical measurements are presented.

On log-concave approximations of high-dimensional posterior measures and stability properties in non-linear inverse problems

Local and global stability properties of the model are identified under which such posterior distributions can be approximated in Wasserstein distance by suitable log-concave measures, which allows the use of fast gradient based sampling algorithms.

On the Computational Complexity of MCMC-Based Estimators in Large Samples

In this paper we examine the implications of the statistical large sample theory for the computational complexity of Bayesian and quasi-Bayesian estimation carried out using Metropolis random walks.…

On some information-theoretic aspects of non-linear statistical inverse problems

Results by van der Vaart (1991) from semi-parametric statistics about the existence of a non-zero Fisher information are reviewed in an infinite-dimensional non-linear Gaussian regression setting.…

Local polynomial kernel regression for generalized linear models and quasi-likelihood functions

Abstract We investigate the extension of the nonparametric regression technique of local polynomial fitting with a kernel weight to generalized linear models and quasi-likelihood contexts. In the…

Theoretical guarantees for approximate sampling from smooth and log‐concave densities

This work establishes non‐asymptotic bounds for the error of approximating the target distribution by the distribution obtained by the Langevin Monte Carlo method and its variants and illustrates the effectiveness of the established guarantees.

CONVERGENCE RATES OF POSTERIOR DISTRIBUTIONS FOR NONIID OBSERVATIONS By

We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general…

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

This work considers approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with non‐Gaussian response variables and can directly compute very accurate approximations to the posterior marginals.

Consistency of Bayesian inference with Gaussian process priors in an elliptic inverse problem

It is shown that, as the number N of measurements increases, the resulting posterior distributions concentrate around the true parameter generating the data, and derive a convergence rate N−λ, λ > 0, for the reconstruction error of the associated posterior means, in L2(O)-distance.

On the Bernstein-von Mises Theorem with Infinite Dimensional Parameters

If there are many independent, identically distributed observations governed by a smooth, finite-dimensional statistical model, the Bayes estimate and the maximum likelihood estimate will be close.…
...