• Corpus ID: 235390877

# Lower Bounds on Metropolized Sampling Methods for Well-Conditioned Distributions

@article{Lee2021LowerBO,
title={Lower Bounds on Metropolized Sampling Methods for Well-Conditioned Distributions},
author={Yin Tat Lee and Ruoqi Shen and Kevin Tian},
journal={ArXiv},
year={2021},
volume={abs/2106.05480}
}
• Published 10 June 2021
• Computer Science, Mathematics
• ArXiv
We give lower bounds on the performance of two of the most popular sampling methods in practice, the Metropolis-adjusted Langevin algorithm (MALA) and multi-step Hamiltonian Monte Carlo (HMC) with a leapfrog integrator, when applied to well-conditioned distributions. Our main result is a nearly-tight lower bound of Ω̃(κd) on the mixing time of MALA from an exponentially warm start, matching a line of algorithmic results [DCWY18, CDWY19, LST20a] up to logarithmic factors and answering an open…

## References

SHOWING 1-10 OF 72 REFERENCES
Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradients
• Computer Science, Mathematics
J. Mach. Learn. Res.
• 2020
This work provides a non-asymptotic upper bound on the mixing time of the Metropolized HMC with explicit choices of stepsize and number of leapfrog steps, and provides a general framework for sharpening mixing time bounds Markov chains initialized at a substantial distance from the target distribution over continuous spaces.
Optimal dimension dependence of the Metropolis-Adjusted Langevin Algorithm
• Computer Science, Mathematics
COLT
• 2021
The upper bound proof introduces a new technique based on a projection characterization of the Metropolis adjustment which reduces the study of MALA to the well-studied discretization analysis of the Langevin SDE and bypasses direct computation of the acceptance probability.
Log-concave sampling: Metropolis-Hastings algorithms are fast!
• Mathematics, Computer Science
COLT
• 2018
A non-asymptotic upper bound on the mixing time of the Metropolis-adjusted Langevin algorithm (MALA) is proved, and the gains of MALA over ULA for weakly log-concave densities are demonstrated.
Dimensionally Tight Bounds for Second-Order Hamiltonian Monte Carlo
• Mathematics, Computer Science
NeurIPS
• 2018
This work shows that the conjecture that Hamiltonian Monte Carlo can be run in gradient evaluations when sampling from strongly log-concave target distributions that satisfy a weak third-order regularity property associated with the input data, and suggests that leapfrog HMC performs better than its competitors when this condition is satisfied.
Theoretical guarantees for approximate sampling from smooth and log‐concave densities
Sampling from various kinds of distributions is an issue of paramount importance in statistics since it is often the key ingredient for constructing estimators, test procedures or confidence
The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo
• Computer Science, Mathematics
J. Mach. Learn. Res.
• 2014
The No-U-Turn Sampler (NUTS), an extension to HMC that eliminates the need to set a number of steps L, and derives a method for adapting the step size parameter {\epsilon} on the fly based on primal-dual averaging.
High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm
• Mathematics, Computer Science
J. Mach. Learn. Res.
• 2021
For a broad class of $d$-dimensional distributions arising from generalized linear models, it is proved that the resulting third-order algorithm produces samples from a distribution that is at most $\varepsilon > 0$ in Wasserstein distance from the target distribution in $O\left(\frac{d^{1/3}}{ \varpsilon^{2/3} \right)$ steps.
Mixing of Hamiltonian Monte Carlo on strongly log-concave distributions 2: Numerical integrators
• Mathematics, Computer Science
AISTATS
• 2019
We obtain quantitative bounds on the mixing properties of the Hamiltonian Monte Carlo (HMC) algorithm with target distribution in d-dimensional Euclidean space, showing that HMC mixes quickly
The geometry of logconcave functions and sampling algorithms
• Computer Science
Random Struct. Algorithms
• 2007
Borders match previous bounds for the special case when the distribution to sample from is the uniform distribution over a convex body, with no assumptions on the local smoothness of the density function.
On sampling from a log-concave density using kinetic Langevin diffusions
• Mathematics, Computer Science
ArXiv
• 2018
It is proved that the geometric mixing property of the kinetic Langevin diffusion with a mixing rate that is, in the overdamped regime, optimal in terms of its dependence on the condition number is optimal.