Optimal rates of entropy estimation over Lipschitz balls

@article{Han2017OptimalRO,
  title={Optimal rates of entropy estimation over Lipschitz balls},
  author={Y. Han and J. Jiao and T. Weissman and Y. Wu},
  journal={ArXiv},
  year={2017},
  volume={abs/1711.02141}
}
  • Y. Han, J. Jiao, +1 author Y. Wu
  • Published 2017
  • Mathematics, Computer Science
  • ArXiv
  • We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates $(n\ln n)^{-\frac{s}{s+d}} + n^{-1/2}$ for $0<s\leq 2$ in arbitrary dimension $d$, where $s$ is the smoothness parameter and $n$ is the number of independent samples. Using a two-stage approximation technique, which first approximate the density by its kernel-smoothed version, and then approximate… CONTINUE READING
    40 Citations
    Minimax estimation of norms of a probability density: I. Lower bounds
    • 1
    • PDF
    On the Estimation of Information Measures of Continuous Distributions
    • 2
    • PDF
    Optimality of the Plug-in Estimator for Differential Entropy Estimation under Gaussian Convolutions
    • 5
    • PDF
    Estimating Differential Entropy under Gaussian Convolutions
    • 9
    • Highly Influenced
    • PDF
    On Estimation of L{r}-Norms in Gaussian White Noise Models
    • 16
    • PDF
    The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
    • 29
    • PDF
    Analysis of KNN Information Estimators for Smooth Distributions
    • Puning Zhao, L. Lai
    • Mathematics, Computer Science
    • IEEE Transactions on Information Theory
    • 2020
    • 1
    • Highly Influenced

    References

    SHOWING 1-10 OF 109 REFERENCES
    Minimax rate-optimal estimation of KL divergence between discrete distributions
    • Y. Han, J. Jiao, T. Weissman
    • Mathematics, Computer Science
    • 2016 International Symposium on Information Theory and Its Applications (ISITA)
    • 2016
    • 30
    • PDF
    On Estimation of L{r}-Norms in Gaussian White Noise Models
    • 16
    • PDF
    Estimation of Nonlinear Functionals of Densities With Confidence
    • 48
    • PDF
    Minimax Estimation of Functionals of Discrete Distributions
    • 193
    • PDF
    Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial Approximation
    • Y. Wu, P. Yang
    • Mathematics, Computer Science
    • IEEE Transactions on Information Theory
    • 2016
    • 175
    • PDF
    The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
    • 29
    • PDF
    Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation
    • 25
    • PDF