Gibbs sampling, exponential families and orthogonal polynomials

@article{Diaconis2008GibbsSE,
  title={Gibbs sampling, exponential families and orthogonal polynomials},
  author={Persi Diaconis and Kshitij Khare and Laurent Saloff-Coste},
  journal={Quality Engineering},
  year={2008},
  volume={54},
  pages={31-32}
}
We give families of examples where sharp rates of convergence to stationarity of the widely used Gibbs sampler are available. The examples involve standard exponential families and their conjugate priors. In each case, the transition operator is explicitly diagonalizable with classical orthogonal polynomials as eigenfunctions. 

Figures from this paper

Gibbs Sampling, Exponential Families and Coupling
We give examples of a quantitative analysis of the bivariate Gibbs sampler using coupling arguments. The examples involve standard statistical models – exponential families with conjugate priors or
Gibbs sampling, conjugate priors and coupling
We give a large family of simple examples where a sharp analysis of the Gibbs sampler can be proved by coupling. These examples involve standard statistical models — exponential families with
Convergence rate and concentration inequalities for Gibbs sampling in high dimension
TLDR
Under the Dobrushin's uniqueness condition, some explicit and sharp estimate of the exponential convergence rate is established and some Gaussian concentration inequalities for the empirical mean are proved.
Stochastic Alternating Projections
The Gibbs sampler, a standard Monte Carlo algorithm, can be seen as alternating projections into closed subspaces of a Hilbert space. This allows classical convergence theorems of von Neumann and
FREE EXPONENTIAL FAMILIES AS KERNEL FAMILIES
  • W. Bryc
  • Mathematics, Computer Science
  • 2006
TLDR
It is shown that free exponential families arise also from a procedure analogous to the definition of exponential families by using the Cauchy-Stieltjes kernel instead of the exponential kernel.
Free Exponential Families as Kernel Families
TLDR
It is shown that free exponential families arise also from the approach analogous to the definition of exponential families by using the Cauchy-Stieltjes kernel 1/(1−θx) instead of the exponential kernel exp( θx).
...
...

References

SHOWING 1-10 OF 190 REFERENCES
Gibbs Sampling, Exponential Families and Coupling
We give examples of a quantitative analysis of the bivariate Gibbs sampler using coupling arguments. The examples involve standard statistical models – exponential families with conjugate priors or
Stochastic Alternating Projections
The Gibbs sampler, a standard Monte Carlo algorithm, can be seen as alternating projections into closed subspaces of a Hilbert space. This allows classical convergence theorems of von Neumann and
Approximation Operators, Exponential, q-Exponential, and Free Exponential Families
Using the technique developed in approximation theory, we construct examples of exponential families of infinitely divisible laws which can be viewed as "-deformations of the normal, gamma, and
Uniform Asymptotics for Polynomials Orthogonal With Respect to a General Class of Discrete Weights and Universality Results for Associated Ensembles
This is the full version of "Uniform Asymptotics for Polynomials Orthogonal With Respect to a General Class of Discrete Weights and Universality Results for Associated Ensembles: Announcement of
Characterization of Markov semigroups on ℝ Associated to Some Families of Orthogonal Polynomials
We give a characterization of the eigenvalues of Markov operators which admit an orthogonal polynomial basis as eigenfunctions, in the Hermite and the Laguerre cases, as well as for the sequences of
Covariance Structure and Convergence Rate of the Gibbs Sampler with Various Scans
This paper presents results on covariance structure and convergence for the Gibbs sampler with both systematic and random scans. It is shown that, under conditions that guarantee the compactness of
Explaining the Gibbs Sampler
TLDR
A simple explanation of how and why the Gibbs sampler works is given and analytically establish its properties in a simple case and insight is provided for more complicated cases.
...
...