Minimax rate-optimal estimation of KL divergence between discrete distributions

@article{Han2016MinimaxRE,
  title={Minimax rate-optimal estimation of KL divergence between discrete distributions},
  author={Yanjun Han and Jiantao Jiao and Tsachy Weissman},
  journal={2016 International Symposium on Information Theory and Its Applications (ISITA)},
  year={2016},
  pages={256-260}
}
We refine the general methodology in [1] for the construction and analysis of essentially minimax estimators for a wide class of functionals of finite dimensional parameters, and elaborate on the case of discrete distributions with support size S comparable with the number of observations n. Specifically, we determine the "smooth" and "non-smooth" regimes based on the confidence set and the smoothness of the functional. In the "non-smooth" regime, we apply an unbiased estimator for a "suitable… CONTINUE READING
Related Discussions
This paper has been referenced on Twitter 5 times. VIEW TWEETS

From This Paper

Figures, tables, and topics from this paper.

Citations

Publications citing this paper.
Showing 1-2 of 2 extracted citations

References

Publications referenced by this paper.
Showing 1-10 of 49 references

Estimation of KL Divergence: Optimal Minimax Rate

IEEE Transactions on Information Theory • 2018
View 5 Excerpts
Highly Influenced

Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial Approximation

IEEE Transactions on Information Theory • 2016
View 6 Excerpts
Highly Influenced

Polynomial Approximation on Polytopes

V. Totik
Memoirs of the American Mathematical Society, • 2013
View 9 Excerpts
Highly Influenced

Minimax Estimation of the L1 Distance

IEEE Trans. Information Theory • 2018
View 10 Excerpts

Estimation of KL divergence between large-alphabet distributions

2016 IEEE International Symposium on Information Theory (ISIT) • 2016
View 1 Excerpt

Optimal estimation of Lr norm of functions in Gaussian white noise

Y. Han, J. Jiao, R. Mukherjee, T. Weissman
preparation, 2016. • 2016
View 3 Excerpts

Similar Papers

Loading similar papers…