# Entropy Computations via Analytic Depoissonization

@article{Jacquet1999EntropyCV, title={Entropy Computations via Analytic Depoissonization}, author={Philippe Jacquet and Wojciech Szpankowski}, journal={IEEE Trans. Inf. Theory}, year={1999}, volume={45}, pages={1072-1081} }

We investigate the basic question of information theory, namely, evaluation of Shannon entropy, and a more general Renyi (1961) entropy, for some discrete distributions (e.g., binomial, negative binomial, etc.). We aim at establishing analytic methods (i.e., those in which complex analysis plays a pivotal role) for such computations which often yield estimates of unparalleled precision. The main analytic tool used here is that of analytic poissonization and depoissonization. We illustrate our…

## 87 Citations

### Entropy computations for discrete distributions: towards analytic information theory

- Mathematics, Computer ScienceProceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252)
- 1998

This work investigates the basic question of information theory, namely, evaluation of Shannon entropy, and a more general Renyi entropy, for some discrete distributions, and aims at establishing analytic methods for such computations which often yield estimates of unparalleled precision.

### Asymptotic average redundancy of Huffman (and other) block codes

- Computer ScienceIEEE Trans. Inf. Theory
- 2000

The appearance of the fractal-like function explains the erratic behavior of the Huffman redundancy, and its "resistance" to succumb to a precise analysis.

### Relative Fisher information of discrete classical orthogonal polynomials

- Mathematics
- 2012

The analytic information theory of discrete distributions was initiated in 1998 by C. Knessl, P. Jacquet and S. Szpankowski who addressed the precise evaluation of the Renyi and Shannon entropies of…

### Sharp Bounds on the Entropy of the Poisson Law and Related Quantities

- Mathematics, Computer ScienceIEEE Transactions on Information Theory
- 2010

Upper and lower bounds for H(¿) are derived that are asymptotically tight and easy to compute and follow easily on the relative entropy D(n, p) between a binomial and a Poisson.

### Analytic variations on redundancy rates of renewal processes

- Computer Science, MathematicsIEEE Trans. Inf. Theory
- 2002

A precise estimate of the redundancy rate for such (nonstationary) renewal sources, namely, 2/(log2)/spl radic/((/spl pi//sup 2//6-1)n)+O(log n) is given, derived by complex-analytic methods that include generating function representations, Mellin transforms, singularity analysis, and saddle-point estimates.

### The Depoissonisation Quintet: Rice-Poisson-Mellin-Newton-Laplace

- MathematicsAofA
- 2018

It is proved that the Rice-Mellin path is both of easy and practical use : even though (much?) less general than the Depoissonisation path, it is easier to apply.

### ASYMPTOTIC ANALYSIS OF PERES’ ALGORITHM FOR RANDOM NUMBER GENERATION

- Mathematics, Computer ScienceProbability in the Engineering and Informational Sciences
- 2022

It is shown that $\lim _{n\to \infty }\log [n-b(n,1/2)]/\log n =\theta =\log [(1+\sqrt {5})/2]$ (where $\log$ is the logarithm to base $2$), which together with limited numerical results suggests that n-b (n, 1/2) may be a regularly varying sequence of index $\theta$.

### Expected External Profile of PATRICIA Tries

- MathematicsANALCO
- 2014

This work considers PATRICIA tries on n random binary strings generated by a memoryless source with parameter p ≥ 1/2 and analyzes asymptotic matching and linearization of the expected value of the external profile at level k = k(n), defined to be the number of leaves atlevel k.

### On Delta-Method of Moments and Probabilistic Sums

- Mathematics, Computer ScienceANALCO
- 2013

A general framework for determining asymptotics of the expected value of random variables of the form f(X) in terms of a function f and central moments of the random variable X and its specific extension to random variables which are sums of identically distributed independent random variables is discussed.

### Maximum Likelihood Estimation of Functionals of Discrete Distributions

- Computer ScienceIEEE Transactions on Information Theory
- 2017

The worst case squared error risk incurred by the maximum likelihood estimator (MLE) in estimating the Shannon entropy is described and it is established that the MLE achieves the minimax optimal rate regardless of the alphabet size.

## References

SHOWING 1-10 OF 39 REFERENCES

### Asymptotic redundancies for universal quantum coding

- Computer ScienceIEEE Trans. Inf. Theory
- 2000

This work compares the quantum asymptotic redundancy formulas derived by naively applying the (nonquantum) counterparts of Clarke and Barren, and finds certain common features.

### Average redundancy rate of the Lempel-Ziv code

- Computer ScienceProceedings of Data Compression Conference - DCC '96
- 1996

It is proved that for a memoryless source the average redundancy rate attains asymptotically Er/sub n/=(A+/spl delta/(n))/log n+O(loglog n/log/sup 2/n) where A is an explicitly given constant that depends on the source characteristics, and /spl delta)/x is a fluctuating function.

### Analysis of digital tries with Markovian dependency

- Computer ScienceIEEE Trans. Inf. Theory
- 1991

A complete characterization of a digital tree is presented from the depth viewpoint in a Markovian framework, that is, under the assumption that symbols in a key are Markov-dependent, and shows that D/sub n/ tends to the normal distribution in all cases except the symmetric independent model.

### Techniques of the Average Case Analysis of Algorithms

- Computer Science, Mathematics
- 1996

This is an extended version of a book chapter that I wrote for the Handbook on Algorithms and Theon) of Computation (Ed. M. Atallah) in which some probabilistic and analytical techniques of the…

### ON ASYMPTOTICS OF CERTAIN RECURRENCES ARISING IN UNIVERSAL CODING

- Computer Science
- 1998

This paper provides asymptotic expansions of certain recurrences studied there which describe the optimal redundancy of universal codes which was recently investigated by Shtarkov et al.

### Asymptotic Behavior of the Lempel-Ziv Parsing Scheme and Digital Search Trees

- Mathematics, Computer ScienceTheor. Comput. Sci.
- 1995

### Average Profile of the Generalized Digital Search Tree and the Generalized Lempel-Ziv Algorithm

- Computer ScienceSIAM J. Comput.
- 1999

The depth of a randomly selected node in a generalized digital search tree and the length of arandomly selected phrase in the generalized Lempel--Ziv parsing scheme are investigated.

### Average profile and limiting distribution for a phrase size in the Lempel-Ziv parsing algorithm

- Computer ScienceIEEE Trans. Inf. Theory
- 1995

The size of a randomly selected phrase, and the average number of phrases of a given size (the so-called average profile of phrase sizes) are focused on.

### Handbook of Combinatorics

- Mathematics
- 1995

Part 1 Structures: graphs - basic graph theory - paths and circuits, J.A. Bondy, connectivity and network flows, A. Frank, matchings and extensions, W.R. Pulleyblank, colouring, stable sets and…