πVAE: a stochastic process prior for Bayesian deep learning with MCMC

@article{Mishra2022VAEAS,
  title={$\pi$VAE: a stochastic process prior for Bayesian deep learning with MCMC},
  author={Swapnil Mishra and Seth Flaxman and Tresnia Berah and Harrison Zhu and Mikko S. Pakkanen and Samir Bhatt},
  journal={Statistics and computing},
  year={2022},
  volume={32 6},
  pages={
          96
        }
}
Stochastic processes provide a mathematically elegant way to model complex data. In theory, they provide flexible priors over function classes that can encode a wide range of interesting assumptions. However, in practice efficient inference by optimisation or marginalisation is difficult, a problem further exacerbated with big data and high dimensional input spaces. We propose a novel variational autoencoder (VAE) called the prior encoding variational autoencoder ( π VAE). π VAE is a new… 

PriorVAE: encoding spatial priors with variational autoencoders for small-area estimation

This work proposes a novel, deep generative modelling approach, termed PriorVAE, which approximate a class of GP priors through prior sampling and subsequent fitting of a variational autoencoder (VAE) and demonstrates the utility of this VAE two-stage approach on Bayesian, small-area estimation tasks.

PriorVAE: Encoding spatial priors with VAEs for small-area estimation

A novel, deep generative modelling approach to tackle spatial inference challenges, termed PriorVAE, which provides tractable and easy-to-implement means of approximately encoding spatial priors and facilitates e-cient statistical inference and demonstrates the utility of the VAE two stage approach on Bayesian, small-area estimation tasks.

Encoding spatiotemporal priors with VAEs for small-area estimation

This work proposes a novel, deep generative modelling approach that provides tractable and easy-to-implement means of approximately encoding spatiotemporal priors and facilitates efficient statistical inference, and demonstrates the utility of the VAE two stage approach on Bayesian, small-area estimation tasks.

Encoding spatial priors with VAEs for small-area estimation

This work proposes a novel, deep generative modelling approach that provides tractable and easy-to-implement means of approximately encoding spatial priors and facilitates e-cient statistical inference and demonstrates the utility of the VAE two stage approach on Bayesian, small-area estimation tasks.

F-EBM: Energy Based Learning of Functional Data

This work presents a novel class of EBM which is able to learn distributions of functions from functional samples evaluated at finitely many points, and has the ability to utilize irregularly sampled training data and output predictions at any resolution.

Neural Diffusion Processes

It is empirically show that NDPs are able to capture functional distributions that are close to the true Bayesian posterior of a Gaussian process, which enables a variety of downstream tasks, including hyperparameter marginalisation and Bayesian optimisation.

Regularised B-splines projected Gaussian Process priors to estimate time-trends of age-specific COVID-19 deaths related to vaccine roll-out (preprint)/ en

The COVID-19 pandemic has caused severe public health consequences in the United States. In this study, we use a hierarchical Bayesian model to estimate the age-specific COVID-19 attributable deaths

Regularised B-splines projected Gaussian Process priors to estimate the age profile of COVID-19 deaths before and after vaccine roll-out

A Bayesian non-parametric spatial approach is used to estimate the age-specific contribution to COVID-19 attributable deaths over time and shows that the spatial approach performs better than a standard B-splines approach and equivalently well as a standard Gaussian Process, for considerably lower runtimes.

Estimating fine age structure and time trends in human contact patterns from coarse contact data: the Bayesian rate consistency model (preprint)

A model-based statistical approach that can reconstruct contact patterns at 1-year resolution even when the age of the contacts is reported coarsely by 5 or 10-year age bands is presented, readily applicable to contemporary survey data as long as the exact age of survey participants is reported.

References

SHOWING 1-10 OF 45 REFERENCES

Hamiltonian Variational Auto-Encoder

It is shown here how to optimally select reverse kernels in this setting and, by building upon Hamiltonian Importance Sampling (HIS), a scheme that provides low-variance unbiased estimators of the ELBO and its gradients using the reparameterization trick is obtained.

Conditional Neural Processes

Conditional Neural Processes are inspired by the flexibility of stochastic processes such as GPs, but are structured as neural networks and trained via gradient descent, yet scale to complex functions and large datasets.

Adam: A Method for Stochastic Optimization

This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.

Spatial mapping with Gaussian processes and nonstationary Fourier features

A Scalable Laplace Approximation for Neural Networks

This work uses recent insights from second-order optimisation for neural networks to construct a Kronecker factored Laplace approximation to the posterior over the weights of a trained network, enabling practitioners to estimate the uncertainty of models currently used in production without having to retrain them.

Stochastic Backpropagation and Approximate Inference in Deep Generative Models

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

This work proposes an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates.

Practical Posterior Error Bounds from Variational Objectives

This paper provides rigorous bounds on the error of posterior mean and uncertainty estimates that arise from full-distribution approximations, as in variational inference.

Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks

This work presents a novel scalable method for learning Bayesian neural networks, called probabilistic backpropagation (PBP), which works by computing a forward propagation of probabilities through the network and then doing a backward computation of gradients.

Bayesian Learning via Stochastic Gradient Langevin Dynamics

In this paper we propose a new framework for learning from large scale datasets based on iterative learning from small mini-batches. By adding the right amount of noise to a standard stochastic