Bregman Power k-Means for Clustering Exponential Family Data

@inproceedings{Vellal2022BregmanPK,
  title={Bregman Power k-Means for Clustering Exponential Family Data},
  author={Adithya D Vellal and Saptarshi Chakraborty and Jason Xu},
  booktitle={ICML},
  year={2022}
}
Recent progress in center-based clustering algorithms combats poor local minima by implicit annealing, using a family of generalized means. These methods are variations of Lloyd’s celebrated k-means algorithm, and are most appropriate for spherical clusters such as those arising from Gaussian data. In this paper, we bridge these algorithmic advances to classical work on hard clustering under Bregman divergences, which enjoy a bijection to exponential family distributions and are thus well… 

Figures and Tables from this paper

References

SHOWING 1-10 OF 41 REFERENCES

Power k-Means Clustering

TLDR
This paper explores an alternative to Lloyd’s algorithm for kmeans clustering that retains its simplicity and mitigates its tendency to get trapped by local minima, and embeds the k-means problem in a continuous class of similar, better behaved problems with fewerLocal minima.

Clustering with Bregman Divergences

TLDR
This paper proposes and analyzes parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences, and shows that there is a bijection between regular exponential families and a largeclass of BRegman diverGences, that is called regular Breg man divergence.

Robust Bregman clustering

TLDR
It is proved that there exists an optimal codebook, and that an empirically optimal codebooks converges a.s. to an optimalcodebook in the distortion sense, and the sub-Gaussian rate of convergence for k-means 1 √ n under mild tail assumptions is obtained.

Entropy Weighted Power k-Means Clustering

TLDR
This paper introduces entropy regularization to learn feature relevance while annealing in k-means and derives a scalable majorization-minimization algorithm that enjoys closed-form updates and convergence guarantees.

A Unified Continuous Optimization Framework for Center-Based Clustering Methods

TLDR
It is demonstrated that within this elementary formulation of the partitioning clustering problem, convex analysis tools and optimization theory provide a unifying language and framework to design, analyze and extend hard and soft center-based clustering algorithms through a generic algorithm which retains the computational simplicity of the popular k-means scheme.

Moment-based Uniform Deviation Bounds for k-means and Friends

TLDR
A soft clustering variant of $k$-means cost is considered, namely the log likelihood of a Gaussian mixture, subject to the constraint that all covariance matrices have bounded spectrum.

On the Surprising Behavior of Distance Metrics in High Dimensional Spaces

TLDR
This paper examines the behavior of the commonly used L k norm and shows that the problem of meaningfulness in high dimensionality is sensitive to the value of k, which means that the Manhattan distance metric is consistently more preferable than the Euclidean distance metric for high dimensional data mining applications.

Bayesian Distance Clustering

TLDR
This work proposes a class of Bayesian distance clustering methods, which rely on modeling the likelihood of the pairwise distances in place of the original data, and illustrates dramatic gains in the ability to infer clusters that are not well represented by the usual choices of kernel.

k-Means clustering with a new divergence-based distance metric: Convergence and performance analysis

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

TLDR
This work describes and analyze an apparatus for adaptively modifying the proximal function, which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal functions that can be chosen in hindsight.