Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction

@article{Cui2016ScalablePA,
  title={Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction},
  author={T. Cui and Y. Marzouk and K. Willcox},
  journal={J. Comput. Phys.},
  year={2016},
  volume={315},
  pages={363-387}
}
Two major bottlenecks to the solution of large-scale Bayesian inverse problems are the scaling of posterior sampling algorithms to high-dimensional parameter spaces and the computational cost of forward model evaluations. Yet incomplete or noisy data, the state variation and parameter dependence of the forward model, and correlations in the prior collectively provide useful structure that can be exploited for dimension reduction in this setting-both in the parameter space of the inverse problem… Expand
Localization for MCMC: sampling high-dimensional posterior distributions with local structure
TLDR
This work investigates how ideas from covariance localization in numerical weather prediction can be used in Markov chain Monte Carlo (MCMC) sampling of high-dimensional posterior distributions arising in Bayesian inverse problems, and discusses the notion of high dimensionality in local problems, which is different from the usual notion of High Dimensionality in function spaceMC. Expand
Certified dimension reduction in nonlinear Bayesian inverse problems
We propose a dimension reduction technique for Bayesian inverse problems with nonlinear forward operators, non-Gaussian priors, and non-Gaussian observation noise. The likelihood function isExpand
Multilevel Dimension-Independent Likelihood-Informed MCMC for Large-Scale Inverse Problems
TLDR
A non-trivial integration of dimension-independent likelihood-informed (DILI) MCMC and the multilevel MCMC to explore the hierarchy of posterior distributions and suggests a new pooling strategy for allocating computational resources across different levels and constructing Markov chains at higher levels conditioned on those simulated on lower levels. Expand
Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions
TLDR
A fast and scalable variational method for Bayesian inference in high-dimensional parameter space, which is called projected Stein variational Newton (pSVN) method, and demonstrates fast convergence of the proposed method and its scalability with respect to the number of parameters, samples, and processor cores. Expand
Localization for MCMC: sampling high-dimensional posterior distributions with banded structure
We investigate how ideas from covariance localization in numerical weather prediction can be used to construct effective Markov chain Monte Carlo (MCMC) methods for sampling high-dimensionalExpand
Stein variational reduced basis Bayesian inversion
TLDR
A Stein variational reduced basis method (SVRB) to solve large-scale PDE-constrained Bayesian inverse problems and develops an adaptive and goal-oriented model reduction technique based on reduced basis approximations for the evaluation of the potential and its gradient. Expand
Hessian-based sampling for high-dimensional model reduction.
TLDR
This work develops a Hessian-based sampling method for the construction of goal-oriented reduced order models with high-dimensional parameter inputs that leads to much smaller errors of the reduced basis approximation for the QoI compared to a random sampling for a diffusion equation with random input obeying either uniform or Gaussian distributions. Expand
Iterative importance sampling algorithms for parameter estimation problems
In parameter estimation problems one approximates a posterior distribution over uncertain param- eters defined jointly by a prior distribution, a numerical model, and noisy data. Typically, MarkovExpand
Bayesian inference with subset simulation in spaces of varying dimension
Uncertainties associated with spatially varying parameters are modeled through random fields discretized into a finite number of random variables. Standard discretization methods, such as theExpand
Scalable Optimization-Based Sampling on Function Space
TLDR
This paper formalizes a function space version of RTO and establishes sufficient conditions for it to produce a valid Metropolis-Hastings proposal, yielding dimension-independent sampling performance and introducing a new subspace acceleration strategy that makes the computational complexity ofRTO scale linearly with the parameter dimension. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 67 REFERENCES
Optimal Low-rank Approximations of Bayesian Linear Inverse Problems
TLDR
Two fast approximations of the posterior mean are proposed and proved optimality with respect to a weighted Bayes risk under squared-error loss and the Hessian of the negative log-likelihood and the prior precision are proved. Expand
Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations
TLDR
It is demonstrated that the number of forward PDE solves required for an accurate low-rank approximation is independent of the problem dimension, which permits scalable estimation of the uncertainty in large-scale ill-posed linear inverse problems at a small multiple of the cost of solving the forward problem. Expand
Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems
TLDR
This work considers a Bayesian approach to nonlinear inverse problems in which the unknown quantity is a spatial or temporal field, endowed with a hierarchical Gaussian process prior, and introduces truncated Karhunen-Loeve expansions, based on the prior distribution, to efficiently parameterize the unknown field. Expand
Dimension-independent likelihood-informed MCMC
TLDR
This work introduces a family of Markov chain Monte Carlo samplers that can adapt to the particular structure of a posterior distribution over functions that may be useful for a large class of high-dimensional problems where the target probability measure has a density with respect to a Gaussian reference measure. Expand
Sparse deterministic approximation of Bayesian inverse problems
We present a parametric deterministic formulation of Bayesian inverse problems with an input parameter from infinite-dimensional, separable Banach spaces. In this formulation, the forward problemsExpand
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion
TLDR
This work addresses the solution of large-scale statistical inverse problems in the framework of Bayesian inference with a so-called Stochastic Monte Carlo method. Expand
A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion
TLDR
A computational framework for estimating the uncertainty in the numerical solution of linearized infinite-dimensional statistical inverse problems that incorporates algorithms for manipulating the prior, constructing a low rank approximation of the data-informed component of the posterior covariance operator, and exploring the posterior that together ensure scalability of the entire framework to very high parameter dimensions. Expand
Sparse-grid, reduced-basis Bayesian inversion
Abstract We analyze reduced basis (RB for short) short acceleration of recently proposed sparse Bayesian inversion algorithms for partial differential equations with uncertain distributed parameter,Expand
A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems, Part II: Stochastic Newton MCMC with Application to Ice Sheet Flow Inverse Problems
TLDR
To address the challenges of sampling high-dimensional pdfs arising from Bayesian inverse problems governed by PDEs, an approximation of the stochastic Newton MCMC method is introduced in which the low-rank-based Hessian is computed at just the MAP point, and then reused at each MCMC step. Expand
Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems
A greedy algorithm for the construction of a reduced model with reduction in both parameter and state is developed for an efficient solution of statistical inverse problems governed by partialExpand
...
1
2
3
4
5
...