• Corpus ID: 10937160

Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility

  title={Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility},
  author={Juho Lee and Seungjin Choi},
Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluation of marginal likelihoods and careful hyperparameter tuning are cumbersome and limit the scalability. In this paper we relax BHC into a non-probabilistic formulation, exploring… 

Figures and Tables from this paper

Inferring Hierarchical Mixture Structures: A Bayesian Nonparametric Approach
A Bayesian Nonparametric model for Hierarchical Clustering and proposes a Markov Chain Monte Carlo procedure with elegantly analytical updating steps for inferring the model variables.
Bayesian Hierarchical Mixture Clustering using Multilevel Hierarchical Dirichlet Processes
A novel Bayesian nonparametric method combining the nested Chinese Restaurant Process (nCRP) and the Hierarchical Dirichlet Process (HDP) is developed, which tackles data with complex latent mixture features which has not been previously explored in the literature.
Small-Variance Asymptotics for Bayesian Models
A recent thread of research has considered small-variance asymptotics of latent-variable models as a way to capture the benefits of rich probabilistic models while also providing a framework for designing more scalable combinatorial optimization algorithms.
Large Sample Asymptotic for Nonparametric Mixture Model with Count Data
The Large Sample Asymptotic for count data when the number of sample in Multinomial distribution goes to infinity is presented and the similar result to SVA for scalable clustering is derived.
A Split-Merge DP-means Algorithm to Avoid Local Minima
An extension of the DP-means algorithm, a hard-clustering approximation of nonparametric Bayesian models, by the split-merge technique is proposed, showing the robustness of the proposed algorithm.
Combinatorial Topic Models using Small-Variance Asymptotics
This work proposes a new objective function derived from LDA by passing to the small-variance limit, and minimize the derived objective by using ideas from combinatorial optimization, which results in a new, fast, and high-quality topic modeling algorithm.
Mapping of Regional-scale Multi-element Geochemical Anomalies Using Hierarchical Clustering Algorithms
Four hierarchical clustering methods are used to identify multi-element geochemical anomalies. The clustering results are confirmed by robust PCA and uni-element geochemical anomalies. The outlined


Revisiting k-means: New Algorithms via Bayesian Nonparametrics
This paper shows that a Gibbs sampling algorithm for the Dirichlet process mixture approaches a hard clustering algorithm in the limit, and further that the resulting algorithm monotonically minimizes an elegant underlying k-means-like clustering objective that includes a penalty for the number of clusters.
Bayesian hierarchical clustering
A novel algorithm for agglomerative hierarchical clustering based on evaluating marginal likelihoods of a probabilistic model that provides a new lower bound on the marginal likelihood of a DPM by summing over exponentially many clusterings of the data in polynomial time.
Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models
This paper derives novel clustering algorithms from the asymptotic limit of the DP and HDP mixtures that features the scalability of existing hard clustering methods as well as the flexibility of Bayesian nonparametric models.
Efficient algorithms for agglomerative hierarchical clustering methods
A centroid SAHN clustering algorithm that requires 0(n2) time, in the worst case, for fixedk and for a family of dissimilarity measures including the Manhattan, Euclidean, Chebychev and all other Minkowski metrics is described.
Clustering with Bregman Divergences
This paper proposes and analyzes parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences, and shows that there is a bijection between regular exponential families and a largeclass of BRegman diverGences, that is called regular Breg man divergence.
An Alternative Prior Process for Nonparametric Bayesian Clustering
This paper investigates the predictive probabilities that underlie the Dirichlet and Pitman-Yor processes, and the implicit "rich-get-richer" characteristic of the resulting partitions, and explores an alternative prior for nonparametric Bayesian clustering|the uniform process|for applications where the rich- get-ricer property is undesirable.
Incremental Tree-Based Inference with Dependent Normalized Random Measures
A tree-based inference method for MNRM mixture models is presented, extending Bayesian hierarchical clustering (BHC) which was originally developed as a deterministic approximate inference for Dirichlet process mixture (DPM) models.
Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems
process. This paper extends Ferguson's result to cases where the random measure is a mixing distribution for a parameter which determines the distribution from which observations are made. The
Hierarchical Grouping to Optimize an Objective Function
Abstract A procedure for forming hierarchical groups of mutually exclusive subsets, each of which has members that are maximally similar with respect to specified characteristics, is suggested for
A Bayesian Analysis of Some Nonparametric Problems
Bayesian approach remained rather unsuccessful in treating nonparametric problems. This is primarily due to the difficulty in finding workable prior distribution on the parameter space , which in