# On empirical cumulant generating functions of code lengths for individual sequences

@article{Merhav2017OnEC, title={On empirical cumulant generating functions of code lengths for individual sequences}, author={Neri Merhav}, journal={2017 IEEE International Symposium on Information Theory (ISIT)}, year={2017}, pages={1500-1504} }

We consider the problem of lossless compression of individual sequences using finite-state (FS) machines, from the perspective of the best achievable empirical cumulant generating function (CGF) of the code length, i.e., the normalized logarithm of the empirical average of the exponentiated code length. Since the probabilistic CGF is minimized in terms of the Renyi entropy of the source, one of the motivations of this study is to derive an individual-sequence analogue of the Renyi entropy, in…

## 2 Citations

### Improved Bounds on Lossless Source Coding and Guessing Moments via Rényi Measures

- Computer ScienceIEEE Transactions on Information Theory
- 2018

Upper and lower bounds on the optimal guessing moments of a random variable taking values on a finite set when side information may be available are provided, similar to Arikan’s bounds, but expressed in terms of the Arimoto-Rényi conditional entropy.

### Fading Channel Coding Based on Entropy and Compressive Sensing

- Computer Science2020 3rd World Symposium on Communication Engineering (WSCE)
- 2020

Channel code length is investigated under Rayleigh and Rician fading assumptions along with additive noise consideration and the inverse problem of identifying the corresponding distributions from the derived channel code lengths and Compressive Sensing based number of samples is addressed.

## References

SHOWING 1-10 OF 13 REFERENCES

### Compression of individual sequences via variable-rate coding

- Computer ScienceIEEE Trans. Inf. Theory
- 1978

The proposed concept of compressibility is shown to play a role analogous to that of entropy in classical information theory where one deals with probabilistic ensembles of sequences rather than with individual sequences.

### Buffer overflow in variable length coding of fixed rate sources

- Computer ScienceIEEE Trans. Inf. Theory
- 1968

In this paper, we develop and analyze an easily instrumentable scheme for variable length encoding of discrete memoryless fixed-rate sources in which buffer overflows result in codeword erasures at…

### Universal coding with minimum probability of codeword length overflow

- Computer ScienceIEEE Trans. Inf. Theory
- 1991

It is shown that the Lempel-Ziv algorithm (1978) asymptotically attains the optimal performance in the sense just defined, independently of the source and the value of B, and faster convergence to theAsymptotic optimum performance can be accomplished by using the minimum-description-length universal code for this subclass.

### The optimal overflow and underflow probabilities with variable-length coding for the general source

- Computer Science2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060)
- 2000

This study shows that the infimum achievable threshold given the overflow probability exponent r always coincides with the inf optimum achievable fixed-length coding rate given the error exponent r, without any assumptions on the source.

### A Measure of Relative Entropy between Individual Sequences with Application to Universal Classification

- Computer ScienceProceedings. IEEE International Symposium on Information Theory
- 1993

A simple universal classification algorithm for individual sequences into a finite number of classes which is based on the empirical divergence, is introduced, which discriminates between the classes whenever they are distinguishable by some finite-memory classifier.

### An inequality on guessing and its application to sequential decoding

- Computer ScienceProceedings of 1995 IEEE International Symposium on Information Theory
- 1995

A simple derivation of the cutoff rate bound for a single-user channels is obtained, and the previously unknown cutoff rate region of multi-access channels is determined.

### Generalization of Huffman coding to minimize the probability of buffer overflow

- Computer ScienceIEEE Trans. Inf. Theory
- 1981

An algorithm is given to find a prefix condition code that minimizes the value of the moment generating function of the codeword length distribution for a given positive argument. This algorithm is…

### Universal decoding for finite-state channels

- Computer ScienceIEEE Trans. Inf. Theory
- 1985

Universal decoding procedures for finite-state channels are discussed. Although the channel statistics are not known, universal decoding can achieve an error probability with an error exponent that,…

### Universal Prediction

- Computer ScienceIEEE Trans. Inf. Theory
- 1998

Both the probabilistic setting and the deterministic setting of the universal prediction problem are described with emphasis on the analogy and the differences between results in the two settings.