Optimal rates of entropy estimation over Lipschitz balls
@article{Han2017OptimalRO, title={Optimal rates of entropy estimation over Lipschitz balls}, author={Y. Han and J. Jiao and T. Weissman and Y. Wu}, journal={ArXiv}, year={2017}, volume={abs/1711.02141} }
We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates $(n\ln n)^{-\frac{s}{s+d}} + n^{-1/2}$ for $0<s\leq 2$ in arbitrary dimension $d$, where $s$ is the smoothness parameter and $n$ is the number of independent samples. Using a two-stage approximation technique, which first approximate the density by its kernel-smoothed version, and then approximate… CONTINUE READING
Topics from this paper
40 Citations
On the Estimation of Information Measures of Continuous Distributions
- Mathematics, Computer Science
- ArXiv
- 2020
- 2
- PDF
Optimality of the Plug-in Estimator for Differential Entropy Estimation under Gaussian Convolutions
- Mathematics, Computer Science
- 2019 IEEE International Symposium on Information Theory (ISIT)
- 2019
- 5
- PDF
On Estimation of L{r}-Norms in Gaussian White Noise Models
- Mathematics, Computer Science
- ArXiv
- 2017
- 16
- PDF
Entropy Rate Estimation for Markov Chains with Large State Space
- Computer Science, Mathematics
- NeurIPS
- 2018
- 10
- PDF
The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
- Computer Science, Mathematics
- NeurIPS
- 2018
- 29
- PDF
The Optimality of Profile Maximum Likelihood in Estimating Sorted Discrete Distributions
- Mathematics, Computer Science
- ArXiv
- 2020
- 1
Analysis of KNN Information Estimators for Smooth Distributions
- Mathematics, Computer Science
- IEEE Transactions on Information Theory
- 2020
- 1
- Highly Influenced
References
SHOWING 1-10 OF 109 REFERENCES
Minimax rate-optimal estimation of KL divergence between discrete distributions
- Mathematics, Computer Science
- 2016 International Symposium on Information Theory and Its Applications (ISITA)
- 2016
- 30
- PDF
On Estimation of L{r}-Norms in Gaussian White Noise Models
- Mathematics, Computer Science
- ArXiv
- 2017
- 16
- PDF
Estimation of Nonlinear Functionals of Densities With Confidence
- Mathematics, Computer Science
- IEEE Transactions on Information Theory
- 2012
- 48
- PDF
Minimax Estimation of Functionals of Discrete Distributions
- Mathematics, Computer Science
- IEEE Transactions on Information Theory
- 2015
- 193
- PDF
Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial Approximation
- Mathematics, Computer Science
- IEEE Transactions on Information Theory
- 2016
- 175
- PDF
The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
- Computer Science, Mathematics
- NeurIPS
- 2018
- 29
- PDF
Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation
- Computer Science, Mathematics
- IEEE Transactions on Information Theory
- 2018
- 25
- PDF