Optimal rates of entropy estimation over Lipschitz balls

@article{Han2017OptimalRO,
title={Optimal rates of entropy estimation over Lipschitz balls},
author={Y. Han and J. Jiao and T. Weissman and Y. Wu},
journal={ArXiv},
year={2017},
volume={abs/1711.02141}
}
• Y. Han, +1 author Y. Wu
• Published 2017
• Mathematics, Computer Science
• ArXiv
• We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates $(n\ln n)^{-\frac{s}{s+d}} + n^{-1/2}$ for $0<s\leq 2$ in arbitrary dimension $d$, where $s$ is the smoothness parameter and $n$ is the number of independent samples. Using a two-stage approximation technique, which first approximate the density by its kernel-smoothed version, and then approximate… CONTINUE READING
40 Citations

Topics from this paper

Optimality of the Plug-in Estimator for Differential Entropy Estimation under Gaussian Convolutions
• Mathematics, Computer Science
• 2019 IEEE International Symposium on Information Theory (ISIT)
• 2019
• 5
• PDF
On Estimation of L{r}-Norms in Gaussian White Noise Models
• Mathematics, Computer Science
• ArXiv
• 2017
• 16
• PDF
Entropy Rate Estimation for Markov Chains with Large State Space
• Computer Science, Mathematics
• NeurIPS
• 2018
• 10
• PDF
Analysis of KNN Information Estimators for Smooth Distributions
• Mathematics, Computer Science
• IEEE Transactions on Information Theory
• 2020
• 1
• Highly Influenced

References

SHOWING 1-10 OF 109 REFERENCES
Minimax rate-optimal estimation of KL divergence between discrete distributions
• Mathematics, Computer Science
• 2016 International Symposium on Information Theory and Its Applications (ISITA)
• 2016
• 30
• PDF
On Estimation of L{r}-Norms in Gaussian White Noise Models
• Mathematics, Computer Science
• ArXiv
• 2017
• 16
• PDF
Estimation of Nonlinear Functionals of Densities With Confidence
• Mathematics, Computer Science
• IEEE Transactions on Information Theory
• 2012
• 48
• PDF
Minimax Estimation of Functionals of Discrete Distributions
• Mathematics, Computer Science
• IEEE Transactions on Information Theory
• 2015
• 193
• PDF
Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial Approximation
• Mathematics, Computer Science
• IEEE Transactions on Information Theory
• 2016
• 175
• PDF
Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation
• Computer Science, Mathematics
• IEEE Transactions on Information Theory
• 2018
• 25
• PDF