Probabilistic numerics and uncertainty in computations

@article{Hennig2015ProbabilisticNA,
  title={Probabilistic numerics and uncertainty in computations},
  author={Philipp Hennig and Michael A. Osborne and Mark A. Girolami},
  journal={Proceedings. Mathematical, Physical, and Engineering Sciences / The Royal Society},
  year={2015},
  volume={471}
}
We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation with limited time or hardware, are important for much contemporary science and industry. Within applications such as climate science and astrophysics, the need to make decisions… 

Figures and Tables from this paper

Probabilistic Integration: A Role in Statistical Computation?
TLDR
These show that probabilistic integrators can in principle enjoy the "best of both worlds", leveraging the sampling efficiency of Monte Carlo methods whilst providing a principled route to assess the impact of numerical error on scientific conclusions.
Probabilistic Integration: A Role for Statisticians in Numerical Analysis?
TLDR
This paper examines thoroughly the case for probabilistic numerical methods in statistical computation and a specific case study is presented for Markov chain and Quasi Monte Carlo methods.
Probabilistic Integration
TLDR
Probabilistic (Bayesian) versions of both Markov chain and Quasi Monte Carlo methods for integration are presented and rigorous theoretical guarantees for convergence rates are provided, in both posterior mean and posterior contraction.
Black Box Probabilistic Numerics
TLDR
This paper proposes to construct probabilistic numerical methods based only on the final output from a traditional method, which massively expands the range of tasks to which Probabilistic numerics can be applied, inherits the features and performance of state-of-the-art numerical methods, and enables provably higher orders of convergence to be achieved.
Probabilistic solvers enable a straight-forward exploration of numerical uncertainty in neuroscience models.
TLDR
It is shown that numerical uncertainty can affect the outcome of typical neuroscience simulations, e.g. jittering spikes by milliseconds or even adding or removing individual spikes from simulations altogether, and it is demonstrated that probabilistic solvers reveal these numerical uncertainties with only moderate computational overhead.
Statistical analysis of differential equations: introducing probability measures on numerical solutions
TLDR
It is shown that a wide variety of existing solvers can be randomised, inducing a probability measure over the solutions of ordinary and partial differential equation models, and the formal means to incorporate this uncertainty in a statistical model and its subsequent analysis are provided.
Bayesian Quadrature for Multiple Related Integrals
TLDR
This paper proposes the first Bayesian probabilistic numerical method by extending the well-known Bayesian quadrature algorithm to the case where it is interested in computing the integral of several related functions, and demonstrates its efficiency in the context of multi-fidelity models for complex engineering systems, as well as a problem of global illumination in computer graphics.
Probability Measures for Numerical Solutions of Differential Equations
In this paper, we present a formal quantification of epistemic uncertainty induced by numerical solutions of ordinary and partial differential equation models. Numerical solutions of differential
A comparison of polynomial chaos and Gaussian process emulation for uncertainty quantification in computer experiments
TLDR
A critical comparison of polynomial chaos and Gaussian process emulation for a range of criteria and examples is provided, with particular focus on the approximation accuracy of the surrogates under changes in the size and type of the experimental design.
Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
TLDR
This paper presents the first probabilistic integrator that admits such theoretical treatment, called Frank-Wolfe Bayesian Quadrature (FWBQ), which is applied to successfully quantify numerical error in the solution to a challenging Bayesian model choice problem in cellular biology.
...
...

References

SHOWING 1-10 OF 68 REFERENCES
Bayesian Uncertainty Quantification for Differential Equations
TLDR
A fully Bayesian inferential framework to quantify uncertainty in models defined by general systems of analytically intractable differential equations, and probabilistically characterises the solution uncertainty introduced when models are chaotic, ill-conditioned, or contain unmodelled functional uncertainty.
Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature
TLDR
A warped model for probabilistic integrands (likelihoods) that are known to be non-negative are introduced, permitting a cheap active learning scheme to optimally select sample locations.
Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition
TLDR
This second edition has been updated and expanded to cover recent developments in applications and theory, including an elegant NP completeness argument by Uwe Naumann and a brief introduction to scarcity, a generalization of sparsity.
Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics
TLDR
This work studies a probabilistic numerical method for the solution of both boundary and initial value problems that returns a joint Gaussian process posterior over the solution that permits marginalising the uncertainty of the numerical solution such that statistics are less sensitive to inaccuracies.
Probabilistic Interpretation of Linear Solvers
TLDR
A probabilistic framework for algorithms that iteratively solve unconstrained linear problems with positive definite $B$ for x with a Gaussian posterior belief over the elements of the inverse of $B$, which can be used to estimate errors.
Active Learning of Model Evidence Using Bayesian Quadrature
TLDR
This work proposes a novel Bayesian Quadrature approach for numerical integration when the integrand is non-negative, such as the case of computing the marginal likelihood, predictive distribution, or normalising constant of a probabilistic model.
Probabilistic ODE Solvers with Runge-Kutta Means
TLDR
A family of probabilistic numerical methods that instead return a Gauss-Markov process defining a probability distribution over the ODE solution, such that posterior means match the outputs of the Runge-Kutta family exactly, thus inheriting their proven good properties.
Bayesian calibration of computer models
TLDR
A Bayesian calibration technique which improves on this traditional approach in two respects and attempts to correct for any inadequacy of the model which is revealed by a discrepancy between the observed data and the model predictions from even the best‐fitting parameter values is presented.
A comparative study of uncertainty propagation methods for black-box-type problems
A wide variety of uncertainty propagation methods exist in literature; however, there is a lack of good understanding of their relative merits. In this paper, a comparative study on the performances
...
...