Maximum Likelihood Estimation of Functionals of Discrete Distributions

@article{Jiao2017MaximumLE,
  title={Maximum Likelihood Estimation of Functionals of Discrete Distributions},
  author={J. Jiao and K. Venkat and Y. Han and T. Weissman},
  journal={IEEE Transactions on Information Theory},
  year={2017},
  volume={63},
  pages={6774-6798}
}
  • J. Jiao, K. Venkat, +1 author T. Weissman
  • Published 2017
  • Mathematics, Computer Science
  • IEEE Transactions on Information Theory
  • We consider the problem of estimating functionals of discrete distributions, and focus on a tight (up to universal multiplicative constants for each specific functional) nonasymptotic analysis of the worst case squared error risk of widely used estimators. We apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true… CONTINUE READING
    78 Citations
    Maximum Likelihood Estimation of information measures
    • 5
    Minimax Estimation of Functionals of Discrete Distributions
    • 193
    • PDF
    Minimax Estimation of Discrete Distributions Under $\ell _{1}$ Loss
    • 42
    • PDF
    Does dirichlet prior smoothing solve the Shannon entropy estimation problem?
    • Y. Han, J. Jiao, T. Weissman
    • Mathematics, Computer Science
    • 2015 IEEE International Symposium on Information Theory (ISIT)
    • 2015
    • 6
    • PDF
    Minimax estimation of discrete distributions
    • Y. Han, J. Jiao, T. Weissman
    • Mathematics, Computer Science
    • 2015 IEEE International Symposium on Information Theory (ISIT)
    • 2015
    • 12
    Ensemble Estimation of Information Divergence †
    • 12
    • PDF
    Minimax estimation of information measures
    • 1
    Chernoff-Type Concentration of Empirical Probabilities in Relative Entropy
    • PDF
    Minimax Optimal Additive Functional Estimation with Discrete Distribution: Slow Divergence Speed Case
    • K. Fukuchi, J. Sakuma
    • Mathematics, Computer Science
    • 2018 IEEE International Symposium on Information Theory (ISIT)
    • 2018
    • PDF

    References

    SHOWING 1-10 OF 106 REFERENCES
    Minimax Estimation of Functionals of Discrete Distributions
    • 193
    • PDF
    Minimax Estimation of KL Divergence between Discrete Distributions
    • 16
    • PDF
    Minimax Estimation of Discrete Distributions Under $\ell _{1}$ Loss
    • 42
    • PDF
    Minimax estimation of discrete distributions
    • Y. Han, J. Jiao, T. Weissman
    • Mathematics, Computer Science
    • 2015 IEEE International Symposium on Information Theory (ISIT)
    • 2015
    • 12
    Bayesian estimation of discrete entropy with mixtures of stick-breaking priors
    • 12
    • PDF
    Bayes' estimators of generalized entropies
    • 41
    • PDF
    Some worst-case bounds for Bayesian estimators of discrete distributions
    • S. Schober
    • Mathematics, Computer Science
    • 2013 IEEE International Symposium on Information Theory
    • 2013
    • 8
    Estimation of Entropy and Mutual Information
    • L. Paninski
    • Mathematics, Computer Science
    • Neural Computation
    • 2003
    • 1,107
    • Highly Influential
    • PDF