Markov Chain Sampling Methods for Dirichlet Process Mixture Models

  title={Markov Chain Sampling Methods for Dirichlet Process Mixture Models},
  author={Radford M. Neal},
  journal={Journal of Computational and Graphical Statistics},
  pages={249 - 265}
  • Radford M. Neal
  • Published 1 June 2000
  • Mathematics
  • Journal of Computational and Graphical Statistics
Abstract This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is to make Metropolis—Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These… 
Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models
This paper describes auxiliary variable representations for the Dirichlet process and the hierarchical Dirichlets that allow us to perform MCMC using the correct equilibrium distribution, in a distributed manner, and shows that this approach allows scalable inference without the deterioration in estimate quality that accompanies existing methods.
A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model
A split-merge Markov chain algorithm is proposed to address the problem of inefficient sampling for conjugate Dirichlet process mixture models by employing a new technique in which an appropriate proposal for splitting or merging components is obtained by using a restricted Gibbs sampling scan.
MCMC for Normalized Random Measure Mixture Models
This paper proposes novel Markov chain Monte Carlo methods for posterior sampling in Bayesian nonparametric mixture models with normalized random measure priors, and describes comparative simulation results demonstrating the efficacies of the proposed methods.
for Normalized Random Measure Mixture Models
Novel Markov chain Monte Carlo methods of both marginal type and conditional type are proposed for posterior sampling in Bayesian nonparametric mixture models with normalized random measure priors.
Posterior Simulation of Normalized Random Measure Mixtures
This article describes posterior simulation methods for mixture models whose mixing distribution has a Normalized Random Measure prior. The methods use slice sampling ideas and introduce no
Sampling the Dirichlet Mixture Model with Slices
  • S. Walker
  • Mathematics, Computer Science
    Commun. Stat. Simul. Comput.
  • 2007
The key to the algorithm detailed in this article, which also keeps the random distribution functions, is the introduction of a latent variable which allows a finite number of objects to be sampled within each iteration of a Gibbs sampler.
Dirichlet Process Mixtures of Generalized Mallows Models
A Dirichlet process mixture model is presented over discrete incomplete rankings and two Gibbs sampling inference techniques for estimating posterior clusterings are studied, demonstrating the effectiveness of this approximation for improving convergence.
Estimation in Dirichlet random effects models
The Gibbs sampler is developed, which is easily extended to a generalized linear mixed model with a probit link function, and is shown to be an improvement, in terms of operator norm and efficiency, over other commonly used MCMC algorithms.
Sequential Quasi Monte Carlo for Dirichlet Process Mixture Models
In mixture models, latent variables known as allocation variables play an essential role by indicating, at each iteration, to which component of the mixture observations are linked. In sequential
Bayesian analysis of random partition models with Laplace distribution
A random partition procedure based on a Dirichlet process prior with Laplace distribution, which provides simultaneous partitioning and parameter estimation with the computation of classification probabilities, unlike its counterparts.


Estimating mixture of dirichlet process models
A conceptual framework for computational strategies is proposed that provides a perspective on current methods, facilitates comparisons between them, and leads to several new methods that expand the scope of MDP models to nonconjugate situations.
Modelling Heterogeneity With and Without the Dirichlet Process
We investigate the relationships between Dirichlet process (DP) based models and allocation models for a variable number of components, based on exchangeable distributions. It is shown that the DP
Computational Methods for Mixture of Dirichlet Process Models
This chapter lays out the basic computational strategies for models based on mixtures of Dirichlet processes and gives advice on how to improve this algorithm through a collapse of the state space of the Markov chain and through blocking of variates for generation.
Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems
process. This paper extends Ferguson's result to cases where the random measure is a mixing distribution for a parameter which determines the distribution from which observations are made. The
Estimating normal means with a conjugate style dirichlet process prior
The problem of estimating many normal means is approached by means of an hierarchical model. The hierarchical model is the standard conjugate model with one exception: the normal distribution at the
Bayesian Density Estimation and Inference Using Mixtures
Abstract We describe and illustrate Bayesian inference in models for density estimation using mixtures of Dirichlet processes. These models provide natural settings for density estimation and are
Computing Nonparametric Hierarchical Models
The ease with which the strict parametric assumptions common to most standard Bayesian hierarchical models can be relaxed to incorporate uncertainties about functional forms using Dirichlet process components is illustrated, partly enabled by the approach to computation using MCMC methods.
Estimating Normal Means with a Dirichlet Process Prior
Abstract In this article, the Dirichlet process prior is used to provide a nonparametric Bayesian estimate of a vector of normal means. In the past there have been computational difficulties with
Sampling Methods For Bayesian Nonparametric Inference Involving Stochastic Processes
An algorithm for simulating the class of neutral to the right processes is described, and a new sampling method, using auxiliary variables, for MDP models is developed and exemplified.