Kullback-Leibler divergence estimation of continuous distributions

@article{PrezCruz2008KullbackLeiblerDE,
  title={Kullback-Leibler divergence estimation of continuous distributions},
  author={Fernando P{\'e}rez-Cruz},
  journal={2008 IEEE International Symposium on Information Theory},
  year={2008},
  pages={1666-1670}
}
We present a method for estimating the KL divergence between continuous densities and we prove it converges almost surely. Divergence estimation is typically solved estimating the densities first. Our main result shows this intermediate step is unnecessary and that the divergence can be either estimated using the empirical cdf or k-nearest-neighbour density estimation, which does not converge to the true measure for finite k. The convergence proof is based on describing the statistics of our… CONTINUE READING

Figures and Topics from this paper.

Citations

Publications citing this paper.
SHOWING 1-10 OF 79 CITATIONS, ESTIMATED 21% COVERAGE

Approximate Bayesian Computation with Kullback-Leibler Divergence as Data Discrepancy

  • AISTATS
  • 2018
VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

co-BPM: a Bayesian Model for Divergence Estimation

VIEW 12 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Distribution Based Spectrum Sensing in Cognitive Radio

VIEW 7 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Physically inspired methods and development of data-driven predictive systems

VIEW 5 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2008
2019

CITATION STATISTICS

  • 12 Highly Influenced Citations

  • Averaged 9 Citations per year over the last 3 years

  • 16% Increase in citations per year in 2018 over 2017

References

Publications referenced by this paper.
SHOWING 1-10 OF 23 REFERENCES

A Nearest-Neighbor Approach to Estimating Divergence between Continuous Random Vectors

  • 2006 IEEE International Symposium on Information Theory
  • 2006
VIEW 6 EXCERPTS
HIGHLY INFLUENTIAL

Estimating mutual information.

  • Physical review. E, Statistical, nonlinear, and soft matter physics
  • 2004
VIEW 1 EXCERPT

Similar Papers

Loading similar papers…