• Corpus ID: 233392742

Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference

@inproceedings{Zhang2021MultiscaleIG,
  title={Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference},
  author={Shumao Zhang and Pengchuan Zhang and Thomas Y. Hou},
  booktitle={International Conference on Machine Learning},
  year={2021}
}
We propose a Multiscale Invertible Generative Network (MsIGN) and associated training algorithm that leverages multiscale structure to solve high-dimensional Bayesian inference. To address the curse of dimensionality, MsIGN exploits the low-dimensional nature of the posterior, and generates samples from coarse to fine scale (low to high dimension) by iteratively upsampling and refining samples. MsIGN is trained in a multistage manner to minimize the Jeffreys divergence, which avoids mode… 

Figures and Tables from this paper

Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint

This paper considers stochastic normalizing flows from a Markov chain point of view, replacing transition densities by general Markov kernels and establishing proofs via Radon-Nikodym derivatives which allows to incorporate distributions without densities in a sound way.

References

SHOWING 1-10 OF 50 REFERENCES

Solving Bayesian inverse problems from the perspective of deep generative networks

This paper investigates their approximation capability in capturing the posterior distribution in Bayesian inverse problems by learning a transport map and proposes a class of network training methods that can be combined with sample-based Bayesian inference algorithms, such as various MCMC algorithms, ensemble Kalman filter and Stein variational gradient descent.

FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models

This paper uses Hutchinson's trace estimator to give a scalable unbiased estimate of the log-density and demonstrates the approach on high-dimensional density estimation, image generation, and variational inference, achieving the state-of-the-art among exact likelihood methods with efficient sampling.

Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions

A fast and scalable variational method for Bayesian inference in high-dimensional parameter space, which is called projected Stein variational Newton (pSVN) method, and demonstrates fast convergence of the proposed method and its scalability with respect to the number of parameters, samples, and processor cores.

A Multiscale Strategy for Bayesian Inference Using Transport Maps

This work introduces a multiscale decomposition that exploits conditional independence across scales, when present in certain classes of inverse problems, to decouple Bayesian inference into two stages: a computationally tractable coarse- scale inference problem, and a mapping of the low-dimensional coarse-scale posterior distribution into the original high-dimensional parameter space.

Projected Stein Variational Gradient Descent

This work proposes a projected Stein variational gradient descent (pSVGD) method to overcome the curse of dimensionality by exploiting the fundamental property of intrinsic low dimensionality of the data informed subspace stemming from ill-posedness of such problems.

Learning to Draw Samples with Amortized Stein Variational Gradient Descent

A simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference based on iteratively adjusting the neural network parameters so that the output changes along a Stein variational gradient direction that maximally decreases the KL divergence with the target distribution.

Residual Flows for Invertible Generative Modeling

The resulting approach, called Residual Flows, achieves state-of-the-art performance on density estimation amongst flow-based models, and outperforms networks that use coupling blocks at joint generative and discriminative modeling.

Density estimation using Real NVP

This work extends the space of probabilistic models using real-valued non-volume preserving (real NVP) transformations, a set of powerful invertible and learnable transformations, resulting in an unsupervised learning algorithm with exact log-likelihood computation, exact sampling, exact inference of latent variables, and an interpretable latent space.

Bayesian Learning via Stochastic Gradient Langevin Dynamics

In this paper we propose a new framework for learning from large scale datasets based on iterative learning from small mini-batches. By adding the right amount of noise to a standard stochastic

Bayesian inference with optimal maps