On the symmetrical Kullback-Leibler Jeffreys centroids

@article{Nielsen2013OnTS,
  title={On the symmetrical Kullback-Leibler Jeffreys centroids},
  author={Frank Nielsen},
  journal={ArXiv},
  year={2013},
  volume={abs/1303.7286}
}
  • F. Nielsen
  • Published 29 March 2013
  • Computer Science
  • ArXiv
Due to the success of the bag-of-word modeling paradigm, clustering histograms has become an important ingredient of modern information processing. Clustering histograms can be performed using the celebrated k-means centroid-based algorithm. From the viewpoint of applications, it is usually required to deal with symmetric distances. In this letter, we consider the Jeffreys divergence that symmetrizes the Kullback-Leibler divergence, and investigate the computation of Jeffreys centroids. We… 

Tables from this paper

On Clustering Histograms with k-Means by Using Mixed α-Divergences

TLDR
This paper investigates the use of a parametric family of distortion measures, called the α-divergences, for clustering histograms, and presents a novel extension of k-means clustering to mixed divergences and reports a guaranteed probabilistic bound.

On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means

TLDR
This work presents a generalization of the Jensen-Shannon (JS) divergence using abstract means which yields closed-form expressions when the mean is chosen according to the parametric family of distributions.

On a Generalization of the Jensen–Shannon Divergence and the Jensen–Shannon Centroid

TLDR
It is proved that the vector-skew α-Jensen–Shannon divergences are f-divergences and it is reported an iterative algorithm to numerically compute the Jensen-Shannon-type centroids for a set of probability densities belonging to a mixture family.

On the Jensen–Shannon Symmetrization of Distances Relying on Abstract Means

TLDR
This work presents a generalization of the Jensen–Shannon (JS) divergence using abstract means which yields closed-form expressions when the mean is chosen according to the parametric family of distributions.

Learning from Both Experts and Data

TLDR
A novel way to tackle the problem of inferring a discrete probability distribution using both expert knowledge and empirical data is presented, with a method providing an objective way to choose the weight to be given to experts compared to data.

Some refelections about the Lambert W function as inverse of x · log(x)

  • U. Tamm
  • Mathematics
    2014 Information Theory and Applications Workshop (ITA)
  • 2014
TLDR
The Lambert W function fulfills W (y)· eW(y) = y and can be applied to invert the function f (x) = x · log(x), which is of some interest in the problems discussed.

Simulation of the south-eastern Polish subpopulations behaviour

TLDR
Using the large database of STR (Short Tandem Repeat) DNA results of south-eastern Polish population, the correctness of data was validated by testing the interdependence of loci and the Hardy-Weinberg hypothesis, and changes of phenotype frequencies could be simulated in the following generations based on some reasonable assumptions.

Proximal methods for convex minimization of Phi-divergences : application to computer vision.

TLDR
This thesis proposes a convex approach to address the problem of dense disparity estimation under varying illumination conditions and proposes a novel approach to handle a broad class of statistical distances, called $varphi-divergences, within the framework of proximal algorithms.

References

SHOWING 1-10 OF 27 REFERENCES

Jeffreys Centroids: A Closed-Form Expression for Positive Histograms and a Guaranteed Tight Approximation for Frequency Histograms

  • F. Nielsen
  • Computer Science
    IEEE Signal Processing Letters
  • 2013
TLDR
The Jeffreys divergence that symmetrizes the Kullback-Leibler divergence is considered, the Jeffreys centroid can be expressed analytically using the Lambert W function for positive histograms, and a fast guaranteed approximation is shown when dealing with frequency histograms.

The centroid of the symmetrical Kullback-Leibler distance

  • R. Veldhuis
  • Computer Science, Mathematics
    IEEE Signal Processing Letters
  • 2002
TLDR
It is shown that the centroid induced by the symmetrical Kullback-Leibler distance is the unique zeroing argument of a function which only depends on the arithmetic and the normalized geometric mean of the cluster.

The Burbea-Rao and Bhattacharyya Centroids

TLDR
An efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods found in the literature that were limited to univariate or “diagonal” multivariate Gaussians.

Sided and Symmetrized Bregman Centroids

  • F. NielsenR. Nock
  • Computer Science, Mathematics
    IEEE Transactions on Information Theory
  • 2009
TLDR
It is proved that all three centroids are unique and give closed-form solutions for the sided centroid that are generalized means, and a provably fast and efficient arbitrary close approximation algorithm for the symmetrized centroid based on its exact geometric characterization is designed.

Using Kullback-Leibler Distance for Text Categorization

TLDR
A new effective model for text categorization with great corpus is introduced, KLD method achieve substantial improvements over the tfidf performing method.

Clustering with Bregman Divergences

TLDR
This paper proposes and analyzes parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences, and shows that there is a bijection between regular exponential families and a largeclass of BRegman diverGences, that is called regular Breg man divergence.

Divergence measures based on the Shannon entropy

TLDR
A novel class of information-theoretic divergence measures based on the Shannon entropy is introduced, which do not require the condition of absolute continuity to be satisfied by the probability distributions involved and are established in terms of bounds.

Visual categorization with bags of keypoints

TLDR
This bag of keypoints method is based on vector quantization of affine invariant descriptors of image patches and shows that it is simple, computationally efficient and intrinsically invariant.

Compressed Histogram of Gradients: A Low-Bitrate Descriptor

TLDR
A framework for computing low bit-rate feature descriptors with a 20× reduction in bit rate compared to state-of-the-art descriptors is proposed and it is shown how to efficiently compute distances between descriptors in the compressed domain eliminating the need for decoding.

ON ASYMPTOTICS OF CERTAIN RECURRENCES ARISING IN UNIVERSAL CODING

TLDR
This paper provides asymptotic expansions of certain recurrences studied there which describe the optimal redundancy of universal codes which was recently investigated by Shtarkov et al.