# Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction

@article{Cui2016ScalablePA, title={Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction}, author={Tiangang Cui and Youssef M. Marzouk and Karen E. Willcox}, journal={J. Comput. Phys.}, year={2016}, volume={315}, pages={363-387} }

Two major bottlenecks to the solution of large-scale Bayesian inverse problems are the scaling of posterior sampling algorithms to high-dimensional parameter spaces and the computational cost of forward model evaluations. Yet incomplete or noisy data, the state variation and parameter dependence of the forward model, and correlations in the prior collectively provide useful structure that can be exploited for dimension reduction in this setting-both in the parameter space of the inverse problem… Expand

#### Figures, Tables, and Topics from this paper

#### 35 Citations

Localization for MCMC: sampling high-dimensional posterior distributions with local structure

- Mathematics, Computer Science
- J. Comput. Phys.
- 2019

This work investigates how ideas from covariance localization in numerical weather prediction can be used in Markov chain Monte Carlo (MCMC) sampling of high-dimensional posterior distributions arising in Bayesian inverse problems, and discusses the notion of high dimensionality in local problems, which is different from the usual notion of High Dimensionality in function spaceMC. Expand

Certified dimension reduction in nonlinear Bayesian inverse problems

- Mathematics
- 2018

We propose a dimension reduction technique for Bayesian inverse problems with nonlinear forward operators, non-Gaussian priors, and non-Gaussian observation noise. The likelihood function is… Expand

Multilevel Dimension-Independent Likelihood-Informed MCMC for Large-Scale Inverse Problems

- Computer Science, Mathematics
- ArXiv
- 2019

A non-trivial integration of dimension-independent likelihood-informed (DILI) MCMC and the multilevel MCMC to explore the hierarchy of posterior distributions and suggests a new pooling strategy for allocating computational resources across different levels and constructing Markov chains at higher levels conditioned on those simulated on lower levels. Expand

Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions

- Computer Science, Mathematics
- NeurIPS
- 2019

A fast and scalable variational method for Bayesian inference in high-dimensional parameter space, which is called projected Stein variational Newton (pSVN) method, and demonstrates fast convergence of the proposed method and its scalability with respect to the number of parameters, samples, and processor cores. Expand

Localization for MCMC: sampling high-dimensional posterior distributions with banded structure

- Mathematics
- 2017

We investigate how ideas from covariance localization in numerical weather prediction can be used to construct effective Markov chain Monte Carlo (MCMC) methods for sampling high-dimensional… Expand

Stein variational reduced basis Bayesian inversion

- Computer Science, Mathematics
- SIAM J. Sci. Comput.
- 2021

A Stein variational reduced basis method (SVRB) to solve large-scale PDE-constrained Bayesian inverse problems and develops an adaptive and goal-oriented model reduction technique based on reduced basis approximations for the evaluation of the potential and its gradient. Expand

HESSIAN-BASED SAMPLING FOR HIGH-DIMENSIONAL MODEL REDUCTION

- Mathematics, Computer Science
- International Journal for Uncertainty Quantification
- 2019

This work develops a Hessian-based sampling method for the construction of goal-oriented reduced order models with high-dimensional parameter inputs that leads to much smaller errors of the reduced basis approximation for the QoI compared to a random sampling for a diffusion equation with random input obeying either uniform or Gaussian distributions. Expand

Iterative importance sampling algorithms for parameter estimation problems

- Computer Science
- 2016

In parameter estimation problems one approximates a posterior distribution over uncertain param- eters defined jointly by a prior distribution, a numerical model, and noisy data. Typically, Markov… Expand

Bayesian inference with subset simulation in spaces of varying dimension

- 2019

Uncertainties associated with spatially varying parameters are modeled through random fields discretized into a finite number of random variables. Standard discretization methods, such as the… Expand

Scalable Optimization-Based Sampling on Function Space

- Computer Science, Mathematics
- SIAM J. Sci. Comput.
- 2020

This paper formalizes a function space version of RTO and establishes sufficient conditions for it to produce a valid Metropolis-Hastings proposal, yielding dimension-independent sampling performance and introducing a new subspace acceleration strategy that makes the computational complexity ofRTO scale linearly with the parameter dimension. Expand

#### References

SHOWING 1-10 OF 67 REFERENCES

Optimal Low-rank Approximations of Bayesian Linear Inverse Problems

- Mathematics, Computer Science
- SIAM J. Sci. Comput.
- 2015

Two fast approximations of the posterior mean are proposed and proved optimality with respect to a weighted Bayes risk under squared-error loss and the Hessian of the negative log-likelihood and the prior precision are proved. Expand

Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations

- Mathematics, Computer Science
- SIAM J. Sci. Comput.
- 2011

It is demonstrated that the number of forward PDE solves required for an accurate low-rank approximation is independent of the problem dimension, which permits scalable estimation of the uncertainty in large-scale ill-posed linear inverse problems at a small multiple of the cost of solving the forward problem. Expand

Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems

- Mathematics, Computer Science
- J. Comput. Phys.
- 2009

This work considers a Bayesian approach to nonlinear inverse problems in which the unknown quantity is a spatial or temporal field, endowed with a hierarchical Gaussian process prior, and introduces truncated Karhunen-Loeve expansions, based on the prior distribution, to efficiently parameterize the unknown field. Expand

Dimension-independent likelihood-informed MCMC

- Mathematics, Computer Science
- J. Comput. Phys.
- 2016

This work introduces a family of Markov chain Monte Carlo samplers that can adapt to the particular structure of a posterior distribution over functions that may be useful for a large class of high-dimensional problems where the target probability measure has a density with respect to a Gaussian reference measure. Expand

Sparse deterministic approximation of Bayesian inverse problems

- Mathematics
- 2012

We present a parametric deterministic formulation of Bayesian inverse problems with an input parameter from infinite-dimensional, separable Banach spaces. In this formulation, the forward problems… Expand

A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion

- Computer Science, Mathematics
- SIAM J. Sci. Comput.
- 2012

This work addresses the solution of large-scale statistical inverse problems in the framework of Bayesian inference with a so-called Stochastic Monte Carlo method. Expand

A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion

- Mathematics, Computer Science
- SIAM J. Sci. Comput.
- 2013

A computational framework for estimating the uncertainty in the numerical solution of linearized infinite-dimensional statistical inverse problems that incorporates algorithms for manipulating the prior, constructing a low rank approximation of the data-informed component of the posterior covariance operator, and exploring the posterior that together ensure scalability of the entire framework to very high parameter dimensions. Expand

Sparse-grid, reduced-basis Bayesian inversion

- Mathematics
- 2015

Abstract We analyze reduced basis (RB for short) short acceleration of recently proposed sparse Bayesian inversion algorithms for partial differential equations with uncertain distributed parameter,… Expand

A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems, Part II: Stochastic Newton MCMC with Application to Ice Sheet Flow Inverse Problems

- Mathematics, Computer Science
- SIAM J. Sci. Comput.
- 2014

To address the challenges of sampling high-dimensional pdfs arising from Bayesian inverse problems governed by PDEs, an approximation of the stochastic Newton MCMC method is introduced in which the low-rank-based Hessian is computed at just the MAP point, and then reused at each MCMC step. Expand

Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems

- Mathematics, Computer Science
- SIAM J. Sci. Comput.
- 2010

A greedy algorithm for the construction of a reduced model with reduction in both parameter and state is developed for an efficient solution of statistical inverse problems governed by partial… Expand