• Corpus ID: 16595212

Weighted Approach to R\'enyi Entropy

  title={Weighted Approach to R\'enyi Entropy},
  author={Marek Śmieja and Jacek Tabor},
Rényi entropy of order α is a general measure of entropy. In this paper we derive estimations for the Rényi entropy of the mixture of sources in terms of the entropy of the single sources. These relations allow to compute the Rényi entropy dimension of arbitrary order of a mixture of measures. The key for obtaining these results is our new definition of the weighted Rényi entropy. It is shown that weighted entropy is equal to the classical Rényi entropy. 


A Coding Theorem and Rényi's Entropy
  • L. Campbell
  • Mathematics, Computer Science
    Inf. Control.
  • 1965
It is shown that L(t) has some desirable properties as a measure of typical code length and a coding theorem for a noiseless channel is proved.
Estimation of the Kolmogorov entropy from a chaotic signal
While there has been recently a dramatic growth in new mathematical concepts related to chaotic systems, ' the detailed comparison between models and experimental data has lagged somewhat. After
Algorithmic information theory
This article is a brief guide to the field of algorithmic information theory, its underlying philosophy, the major subfields, applications, history, and a map of the field are presented.
Entropy and Information Theory
  • R. Gray
  • Mathematics
    Springer New York
  • 1990
This book is an updated version of the information theory classic, first published in 1990. About one-third of the book is devoted to Shannon source and channel coding theorems; the remainder
Rényi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression
  • Yihong Wu, S. Verdú
  • Mathematics, Computer Science
    IEEE Transactions on Information Theory
  • 2010
Almost lossless analog compression for analog memoryless sources in an information-theoretic framework, in which the compressor or decompressor is constrained by various regularity conditions, in particular linearity of the compressor and Lipschitz continuity of the decompressor.
Epsilon entropy of stochastic processes.
Epsilon-delta entropy for probabilistic metric spaces applied to stochastic processes in data transmission
A Mathematical Theory of Communication
This paper opened the new area the information theory. Before this paper, most people believed that the only way to make the error probability of transmission as small as desired is to reduce the
Entropy of the Mixture of Sources and Entropy Dimension
The basic role is played by a new equivalent definition of entropy based on measures instead of partitions, which decomposes the entropy dimension of the mixture of sources in terms of the convex combination of the entropy dimensions of the single sources.
Entropy, large deviations, and statistical mechanics
I: Large Deviations and Statistical Mechanics.- I. Introduction to Large Deviations.- I.1. Overview.- I.2. Large Deviations for I.I.D. Random Variables with a Finite State Space.- I.3. Levels-1 and 2
Convex Functions and Their Applications: A Contemporary Approach
This second edition provides a thorough introduction to contemporary convex function theory with many new results. A large variety of subjects are covered, from the one real variable case to some o