Variance reduction for Markov chains with application to MCMC

  title={Variance reduction for Markov chains with application to MCMC},
  author={Denis Belomestny and Leonid Iosipoi and {\'E}ric Moulines and Alexey Naumov and Sergey Samsonov},
  journal={Statistics and Computing},
In this paper, we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce the overall finite sample variance. This feature is theoretically demonstrated by means of a deep non-asymptotic analysis of a variance reduced functional as well as by a thorough… 
Variance reduction for additive functionals of Markov chains via martingale representations
By rigorously analyzing the convergence properties of the proposed algorithm, it is shown that its cost-to-variance product is indeed smaller than one of the naive algorithms.
Unbiased Markov chain Monte Carlo methods with couplings
The theoretical validity of the estimators proposed and their efficiency relative to the underlying MCMC algorithms are established and the performance and limitations of the method are illustrated.
Variance Reduction for Metropolis-Hastings Samplers
We introduce a general framework that constructs estimators with reduced variance for random walk Metropolis and Metropolis-adjusted Langevin algorithms. The resulting estimators require negligible
Scalable Control Variates for Monte Carlo Methods via Stochastic Optimization
This paper considers control variates based on Stein operators, presenting a framework that encompasses and generalizes existing approaches that use polynomials, kernels and neural networks, leading to scalable and effective control Variates.
Postprocessing of MCMC
Markov chain Monte Carlo is the engine of modern Bayesian statistics, being used to approximate the posterior and derived quantities of interest. Despite this, the issue of how the output from a
Variance Reduction for Dependent Sequences with Applications to Stochastic Gradient MCMC
This work proposes a novel and practical variance reduction approach for additive functionals of dependent sequences that combines the use of control variates with the minimization of randomness in these sequences.
Stein's Method Meets Computational Statistics: A Review of Some Recent Developments
Stein’s method compares probability distributions through the study of a class of linear operators called Stein operators. While mainly studied in probability and used to underpin theoretical
Control variate selection for Monte Carlo integration
Regularizing the ordinary least squares estimator by preselecting appropriate control variates via the Lasso turns out to increase the accuracy without additional computational cost.
Vector-Valued Control Variates
Control variates are post-processing tools for Monte Carlo estimators which can lead to significant variance reduction. This approach usually requires a large number of samples, which can be
Semi-Exact Control Functionals From Sard’s Method
A novel control variate technique is proposed for post-processing of Markov chain Monte Carlo output, based both on Stein’s method and an approach to numerical integration due to Sard. The


Diffusion approximations and control variates for MCMC
A new methodology is presented for the construction of control variates to reduce the variance of additive functionals of Markov Chain Monte Carlo (MCMC) samplers, and provides an explicit representation of the optimal coefficients minimizing the asymptotic variance of the Langevin diffusion.
Zero variance Markov chain Monte Carlo for Bayesian estimators
A general purpose variance reduction technique for the MCMC estimator, based on the zero-variance principle introduced in the physics literature, is proposed, and conditions for asymptotic unbiasedness of the Zero Variance estimator are derived.
Control variates for estimation based on reversible Markov chain Monte Carlo samplers
Summary.  A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible Markov chain Monte Carlo
Batch means and spectral variance estimators in Markov chain Monte Carlo
Calculating a Monte Carlo standard error (MCSE) is an important step in the statistical analysis of the simulation output obtained from a Markov chain Monte Carlo experiment. An MCSE is usually based
Control functionals for Monte Carlo integration
A non‐parametric extension of control variates is presented. These leverage gradient information on the sampling density to achieve substantial variance reduction. It is not required that the
Convergence rates for a class of estimators based on Stein’s method
Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a
Zero Variance Differential Geometric Markov Chain Monte Carlo Algorithms
This paper suggests that part of the additional computing required by Hamiltonian Monte Carlo and Metropolis adjusted Langevin algorithms produces elements that allow concurrent implementation of the zero variance reduction technique for MCMC estimation.
Regularised Zero-Variance Control Variates for High-Dimensional Variance Reduction
Zero-variance control variates (ZV-CV) are a post-processing method to reduce the variance of Monte Carlo estimators of expectations using the derivatives of the log target. Once the derivatives are
General state space Markov chains and MCMC algorithms
This paper surveys various results about Markov chains on gen- eral (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the
Variance Reduction in Monte Carlo Estimators via Empirical Variance Minimization
For Monte Carlo estimators, a variance reduction method based on empirical variance minimization in the class of functions with zero mean (control functions) is described. An upper bound for the