# Entropy Computations via Analytic Depoissonization

@article{Jacquet1999EntropyCV,
title={Entropy Computations via Analytic Depoissonization},
author={Philippe Jacquet and Wojciech Szpankowski},
journal={IEEE Trans. Inf. Theory},
year={1999},
volume={45},
pages={1072-1081}
}
• Published 1 May 1999
• Computer Science, Mathematics
• IEEE Trans. Inf. Theory
We investigate the basic question of information theory, namely, evaluation of Shannon entropy, and a more general Renyi (1961) entropy, for some discrete distributions (e.g., binomial, negative binomial, etc.). We aim at establishing analytic methods (i.e., those in which complex analysis plays a pivotal role) for such computations which often yield estimates of unparalleled precision. The main analytic tool used here is that of analytic poissonization and depoissonization. We illustrate our…

## Figures from this paper

• Mathematics, Computer Science
Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252)
• 1998
This work investigates the basic question of information theory, namely, evaluation of Shannon entropy, and a more general Renyi entropy, for some discrete distributions, and aims at establishing analytic methods for such computations which often yield estimates of unparalleled precision.
The appearance of the fractal-like function explains the erratic behavior of the Huffman redundancy, and its "resistance" to succumb to a precise analysis.
• Mathematics
• 2012
The analytic information theory of discrete distributions was initiated in 1998 by C. Knessl, P. Jacquet and S. Szpankowski who addressed the precise evaluation of the Renyi and Shannon entropies of
• Mathematics, Computer Science
IEEE Transactions on Information Theory
• 2010
Upper and lower bounds for H(¿) are derived that are asymptotically tight and easy to compute and follow easily on the relative entropy D(n, p) between a binomial and a Poisson.
• Computer Science, Mathematics
IEEE Trans. Inf. Theory
• 2002
A precise estimate of the redundancy rate for such (nonstationary) renewal sources, namely, 2/(log2)/spl radic/((/spl pi//sup 2//6-1)n)+O(log n) is given, derived by complex-analytic methods that include generating function representations, Mellin transforms, singularity analysis, and saddle-point estimates.
It is proved that the Rice-Mellin path is both of easy and practical use : even though (much?) less general than the Depoissonisation path, it is easier to apply.
• Mathematics, Computer Science
Probability in the Engineering and Informational Sciences
• 2022
It is shown that $\lim _{n\to \infty }\log [n-b(n,1/2)]/\log n =\theta =\log [(1+\sqrt {5})/2]$ (where $\log$ is the logarithm to base $2$), which together with limited numerical results suggests that n-b (n, 1/2) may be a regularly varying sequence of index $\theta$.
• Mathematics
ANALCO
• 2014
This work considers PATRICIA tries on n random binary strings generated by a memoryless source with parameter p ≥ 1/2 and analyzes asymptotic matching and linearization of the expected value of the external profile at level k = k(n), defined to be the number of leaves atlevel k.
• Mathematics, Computer Science
ANALCO
• 2013
A general framework for determining asymptotics of the expected value of random variables of the form f(X) in terms of a function f and central moments of the random variable X and its specific extension to random variables which are sums of identically distributed independent random variables is discussed.
• Computer Science
IEEE Transactions on Information Theory
• 2017
The worst case squared error risk incurred by the maximum likelihood estimator (MLE) in estimating the Shannon entropy is described and it is established that the MLE achieves the minimax optimal rate regardless of the alphabet size.

## References

SHOWING 1-10 OF 39 REFERENCES

• Computer Science
IEEE Trans. Inf. Theory
• 2000
This work compares the quantum asymptotic redundancy formulas derived by naively applying the (nonquantum) counterparts of Clarke and Barren, and finds certain common features.
• Computer Science
Proceedings of Data Compression Conference - DCC '96
• 1996
It is proved that for a memoryless source the average redundancy rate attains asymptotically Er/sub n/=(A+/spl delta/(n))/log n+O(loglog n/log/sup 2/n) where A is an explicitly given constant that depends on the source characteristics, and /spl delta)/x is a fluctuating function.
• Computer Science
IEEE Trans. Inf. Theory
• 1991
A complete characterization of a digital tree is presented from the depth viewpoint in a Markovian framework, that is, under the assumption that symbols in a key are Markov-dependent, and shows that D/sub n/ tends to the normal distribution in all cases except the symmetric independent model.
This is an extended version of a book chapter that I wrote for the Handbook on Algorithms and Theon) of Computation (Ed. M. Atallah) in which some probabilistic and analytical techniques of the
This paper provides asymptotic expansions of certain recurrences studied there which describe the optimal redundancy of universal codes which was recently investigated by Shtarkov et al.
• Computer Science
SIAM J. Comput.
• 1999
The depth of a randomly selected node in a generalized digital search tree and the length of arandomly selected phrase in the generalized Lempel--Ziv parsing scheme are investigated.
• Computer Science
IEEE Trans. Inf. Theory
• 1995
The size of a randomly selected phrase, and the average number of phrases of a given size (the so-called average profile of phrase sizes) are focused on.
• Mathematics
• 1995
Part 1 Structures: graphs - basic graph theory - paths and circuits, J.A. Bondy, connectivity and network flows, A. Frank, matchings and extensions, W.R. Pulleyblank, colouring, stable sets and