Optimal scaling of discrete approximations to Langevin diffusions

@article{Roberts1998OptimalSO,
  title={Optimal scaling of discrete approximations to Langevin diffusions},
  author={Gareth O. Roberts and Jeffrey S. Rosenthal},
  journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
  year={1998},
  volume={60}
}
  • G. Roberts, J. Rosenthal
  • Published 1998
  • Computer Science, Mathematics
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology)
We consider the optimal scaling problem for proposal distributions in Hastings–Metropolis algorithms derived from Langevin diffusions. We prove an asymptotic diffusion limit theorem and show that the relative efficiency of the algorithm can be characterized by its overall acceptance rate, independently of the target distribution. The asymptotically optimal acceptance rate is 0.574. We show that, as a function of dimension n, the complexity of the algorithm is O(n1/3), which compares favourably… 

Fast Langevin based algorithm for MCMC in high dimensions

TLDR
An asymptotic diffusion limit theorem is proved and it is shown that the relative efficiency of the Hastings-Metropolis algorithm can be characterised by its overall acceptance rate (with asymPTotical value 0.704), independently of the target distribution.

Optimal dimension dependence of the Metropolis-Adjusted Langevin Algorithm

TLDR
The upper bound proof introduces a new technique based on a projection characterization of the Metropolis adjustment which reduces the study of MALA to the well-studied discretization analysis of the Langevin SDE and bypasses direct computation of the acceptance probability.

Optimal Scaling and Diffusion Limits for the Langevin Algorithm in High Dimensions

TLDR
It is proved that, started in stationarity, a suitably interpolated and scaled version of the Markov chain corresponding to MALA converges to an infinite dimensional diffusion process.

Optimal scalings for local Metropolis--Hastings chains on nonproduct targets in high dimensions

We investigate local MCMC algorithms, namely the random-walk Metropolis and the Langevin algorithms, and identify the optimal choice of the local step-size as a function of the dimension $n$ of the

Langevin Diffusions and Metropolis-Hastings Algorithms

We consider a class of Langevin diffusions with state-dependent volatility. The volatility of the diffusion is chosen so as to make the stationary distribution of the diffusion with respect to its

Weak Convergence and Optimal Proposals for Monte Carlo Algorithms

TLDR
These notes summarize most of the literature regarding the optimal tuning of Markov chain Monte Carlo algorithms, and discuss the applications of these findings in practice.

Optimal metropolis algorithms for product measures on the vertices of a hypercube

TLDR
This paper considers the simplest possible (and most discrete) example of such a problem, demonstrating that, at least for sufficiently 'smooth' distributions in high dimensional problems, the Metropolis algorithm behaves similarly to its counterpart on the continuous state space.

Scaling limits for the transient phase of local Metropolis–Hastings algorithms

Summary.  The paper considers high dimensional Metropolis and Langevin algorithms in their initial transient phase. In stationarity, these algorithms are well understood and it is now well known how

Optimal Scaling for Random Walk Metropolis on Spherically Constrained Target Densities

We consider the problem of optimal scaling of the proposal variance for multidimensional random walk Metropolis algorithms. It is well known, for a wide range of continuous target densities, that the

Diffusion limits of the random walk metropolis algorithm in high dimensions

Diffusion limits of MCMC methods in high dimensions provide a useful theoretical tool for studying computational complexity. In particular, they lead directly to precise estimates of the number of
...

References

SHOWING 1-10 OF 24 REFERENCES

Weak convergence and optimal scaling of random walk Metropolis algorithms

This paper considers the problem of scaling the proposal distribution of a multidimensional random walk Metropolis algorithm in order to maximize the efficiency of the algorithm. The main result is a

Exponential Convergence of Langevin Diiusions and Their Discrete Approximations

In this paper we consider a continous time method of approximating a given distribution using the Langevin diiusion dL t = dW t + 1 2 r log (L t)dt: We nd conditions under which this diiusion

Acceptances and autocorrelations in hybrid Monte Carlo

Brownian dynamics as smart Monte Carlo simulation

A new Monte Carlo simulation procedure is developed which is expected to produce more rapid convergence than the standard Metropolis method. The trial particle moves are chosen in accord with a

Monte Carlo Sampling Methods Using Markov Chains and Their Applications

SUMMARY A generalization of the sampling method introduced by Metropolis et al. (1953) is presented along with an exposition of the relevant theory, techniques of application and methods and

Probability: Theory and Examples

This book is an introduction to probability theory covering laws of large numbers, central limit theorems, random walks, martingales, Markov chains, ergodic theorems, and Brownian motion. It is a

Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus

The use of the Gibbs sampler for Bayesian computation is reviewed and illustrated in the context of some canonical examples. Other Markov chain Monte Carlo simulation methods are also briefly

Equation of state calculations by fast computing machines

A general method, suitable for fast computing machines, for investigating such properties as equations of state for substances consisting of interacting individual molecules is described. The method

Mathematica - a system for doing mathematics by computer, 2nd Edition

TLDR
This new edition maintains the format of the original book and is the single most important user guide and reference for Mathematica--all users ofMathematica will need this edition.