A general framework for codes involving redundancy minimization

@article{Baer2006AGF,
  title={A general framework for codes involving redundancy minimization},
  author={Michael B. Baer},
  journal={IEEE Transactions on Information Theory},
  year={2006},
  volume={52},
  pages={344-349}
}
  • M. Baer
  • Published 18 August 2005
  • Computer Science
  • IEEE Transactions on Information Theory
A framework with two scalar parameters is introduced for various problems of finding a prefix code minimizing a coding penalty function. The framework encompasses problems previously proposed by Huffman, Campbell, Nath, and Drmota and Szpankowski, shedding light on the relationships among these problems. In particular, Nath's range of problems can be seen as bridging the minimum average redundancy problem of Huffman with the minimum maximum pointwise redundancy problem of Drmota and Szpankowski… 

Figures from this paper

Minimum Redundancy Coding for Uncertain Sources
TLDR
This paper examines the related problem of maximal minimax pointwise redundancy and the problem considered by Gawrychowski and Gagie, which, for a sufficiently small relative entropy ball, is equivalent to minimax redundancy.
Redundancy-Related Bounds for Generalized Huffman Codes
  • M. Baer
  • Computer Science
    IEEE Transactions on Information Theory
  • 2011
TLDR
New lower and upper bounds for the compression rate of binary prefix codes optimized over memoryless sources according to various nonlinear codeword length objectives are presented, which relate to various problems involving queueing, uncertainty, and lossless communications.
Redundancy-Related Bounds for
TLDR
New lower and upper bounds for the compression rate of binary prefix codes optimized over memo- ryless sources according to various nonlinear codeword length ob- jectives are presented, which relates to various problems involving queueing, un- certainty, and lossless communications.
Bounds on Generalized Huffman Codes
TLDR
New lower and upper bounds are obtained for the compression of optimal binary prefix codes according to various nonlinear codeword length objectives in terms of a form of entropy and the probability of the most probable input symbol.
Lossless Coding with Generalised Criteria
TLDR
This framework encompasses as a special case several criteria previously investigated in the literature, while relations to universal coding is discussed, including a convex combination of the average of an exponential function of the codeword length and the average redundancy.
Lossless Source Codes for a Class Described by Variational Distance and Optimal Merging Algorithm
AbstractThis paper considers lossless uniquely decodable source codes for a class of distributions describedby a ball with respect to the total variational distance, centered at a nominal
Optimal Merging Algorithms for Lossless Codes With Generalized Criteria
TLDR
This paper presents lossless prefix codes optimized with respect to a payoff criterion consisting of a convex combination of maximum codeword length and average codewords length based on a new coding algorithm, which transforms the initial source probability vector into a new probability vector according to a merging rule.
Optimal Prefix Codes for Infinite Alphabets With Nonlinear Costs
  • M. Baer
  • Computer Science
    IEEE Transactions on Information Theory
  • 2008
TLDR
Methods for finding codes optimal for beta-exponential means are introduced and one method applies to geometric distributions, while another applies to distributions with lighter tails and both are extended to alphabetic codes.
Variable Length Lossless Coding for Variational Distance Class: An Optimal Merging Algorithm
TLDR
This paper considers lossless source coding for a class of sources specified by the total variational distance ball centred at a fixed nominal probability distribution, and examines the maximization of the average codeword length by converting it into an equivalent optimization problem, and gives the optimal codewords via a waterfilling solution.
Lossless coding with generalized criteria
TLDR
This framework encompasses as a special case several criteria previously investigated in the literature, while relations to universal coding is discussed, and an algorithm is presented to compute the weighting vector.
...
...

References

SHOWING 1-10 OF 53 REFERENCES
Generalized Shannon Code Minimizes the Maximal Redundancy
TLDR
The main findings of this paper are established by techniques that belong to the toolkit of the "analytic analysis of algorithms" such as theory of distribution of sequences modulo 1 and Fourier series, which have already found applications in other problems of information theory.
Precise minimax redundancy and regret
TLDR
This work proves that a generalized Shannon code minimizes the worst case redundancy, derive asymptotically its redundancy, and establish some general properties, and presents the exact constant of the redundancy for memoryless and Markov sources.
The Rényi redundancy of generalized Huffman codes
TLDR
By decreasing some of the codeword lengths in a Shannon code, the upper bound on the redundancy given in the standard proof of the noiseless source coding theorem is improved and the lower bound is improved by randomizing between codewords, allowing linear programming techniques to be used on an integer programming problem.
Minimum Variance Huffman Codes
TLDR
The minimum redundancy code with the minimum variance of the word length is characterized and it is shown that it is in a certain sense unique and has a strong property in that it minimizes a general class of functions of the minimum redundancy codes.
Maximum Distance Separable Convolutional Codes
TLDR
The main result of the paper shows that this upper bound for the free distance generalizing the Singleton bound can be achieved in all cases if one allows sufficiently many field elements.
Minimal Huffman trees
TLDR
It is proved that several natural variants of Huffman's algorithm, that appear to be nondeterministic, in fact all lead to the single Huffman tree obtained by Schwartz's algorithm.
A Short Note on the Redundancy of Degree alpha
A method for the construction of minimum-redundancy codes
  • D. Huffman
  • Computer Science, Business
    Proceedings of the IRE
  • 1952
TLDR
A minimum-redundancy code is one constructed in such a way that the average number of coding digits per message is minimized.
An Optimum Encoding with Minimum Longest Code and Total Number of Digits
...
...