# Mixture Models With a Prior on the Number of Components

@article{Miller2015MixtureMW,
title={Mixture Models With a Prior on the Number of Components},
author={Jeffrey W. Miller and Matthew T. Harrison},
journal={Journal of the American Statistical Association},
year={2015},
volume={113},
pages={340 - 356}
}
• Published 22 February 2015
• Computer Science
• Journal of the American Statistical Association
ABSTRACT A natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with symmetric Dirichlet weights, and put a prior on the number of components—that is, to use a mixture of finite mixtures (MFM). The most commonly used method of inference for MFMs is reversible jump Markov chain Monte Carlo, but it can be nontrivial to design good reversible jump moves, especially in high-dimensional spaces. Meanwhile, there are samplers for…
175 Citations
R rigor is added to data-analysis folk wisdom by proving that under even the slightest model misspecification, the FMM posterior on the number of components is ultraseverely inconsistent: for any finite $k \in \mathbb{N}$, the posterior probability that the numberof components is $k$ converges to 0 in the limit of infinite data.
• Computer Science, Mathematics
• 2022
It is shown that a post-processing algorithm introduced by Guha et al. (2021) for the Dirichlet process extends to more general models and provides a consistent method to estimate the number of components and discusses possible solutions.
A new class of priors is introduced: the Normalized Independent Point Process, which is based on an auxiliary variable MCMC, which allows handling the otherwise intractable posterior distribution and overcomes the challenges associated with the Reversible Jump algorithm.
• Computer Science, Mathematics
• 2021
This work derives a sampler that is straightforward to implement for mixing distributions with tractable size-biased ordered weights and mitigates the label-switching problem in infinite mixtures.
• Computer Science
Journal of Computational and Graphical Statistics
• 2021
A general framework for mixture models, when the prior of the “cluster centers” is a finite repulsive point process depending on a hyperparameter, specified by a density which may depend on an intractable normalizing constant is presented.
• Computer Science
• 2022
The results show that the choice of prior is critical for deriving reliable posterior inferences in problems of higher dimensionality, and the use of the DPMM in clustering is also applicable to density estimation.
• Mathematics, Computer Science
Biometrika
• 2022
This work focuses on consistency for the unknown number of clusters when the observed data are generated from a finite mixture, and considers the situation where a prior is placed on the concentration parameter of the underlying Dirichlet process.
• Mathematics
• 2020
Discrete nonparametric priors play a central role in a variety of Bayesian procedures, most notably when used to model latent features as in clustering, mixtures and curve fitting. They are effective
• Computer Science
Bernoulli
• 2021
It will be shown that the modeling choice of kernel density functions plays perhaps the most impactful roles in determining the posterior contraction rates in the misspecified situations.
This thesis shows how to overcome certain intractabilities in order to obtain analogous compact representations for the class of Poisson-Kingman priors which includes the Dirichlet and Pitman-Yor processes.

## References

SHOWING 1-10 OF 136 REFERENCES

• Computer Science
J. Mach. Learn. Res.
• 2014
It is shown that the posterior on data from a finite mixture does not concentrate at the true number of components, and this result applies to a large class of nonparametric mixtures, including DPMs and PYMs, over a wide variety of families of component distributions.
• Computer Science, Mathematics
NIPS
• 2013
An elementary proof of this inconsistency is given in what is perhaps the simplest possible setting: a DPM with normal components of unit variance, applied to data from a "mixture" with one standard normal component.
• Computer Science
• 2006
A variational inference algorithm forDP mixtures is presented and experiments that compare the algorithm to Gibbs sampling algorithms for DP mixtures of Gaussians and present an application to a large-scale image analysis problem are presented.
• Computer Science, Mathematics
• 2001
A weighted Bayes factor method for consistently estimating d that can be implemented by an iid generalized weighted Chinese restaurant (GWCR) Monte Carlo algorithm and the performance of the new GWCR model selection procedure is compared with that of the Akaike information criterion and the Bayes information criterion implemented through an EM algorithm.
• Mathematics
• 2005
In recent years the Dirichlet process prior has experienced a great success in the context of Bayesian mixture modeling. The idea of overcoming discreteness of its realizations by exploiting it in
• Mathematics
Stat. Comput.
• 2011
A more efficient version of the slice sampler for Dirichlet process mixture models described by Walker allows for the fitting of infinite mixture models with a wide-range of prior specifications and considers priors defined through infinite sequences of independent positive random variables.
• Mathematics
• 1997
New methodology for fully Bayesian mixture analysis is developed, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces
Finite mixture distributions are receiving more and more attention from statisticians in many different fields of research because they are a very flexible class of models. They are typically used
• Mathematics
• 2003
The class of species sampling mixture models is introduced as an exten- sion of semiparametric models based on the Dirichlet process to models based on the general class of species sampling priors,
In Bayesian density estimation and prediction using Dirichlet process mixtures of standard, exponential family distributions, the precision or total mass parameter of the mixing Dirichlet process is