• Corpus ID: 702542

Elements of information theory (2. ed.)

  title={Elements of information theory (2. ed.)},
  author={Thomas M. Cover and Joy A. Thomas},


  • H. Ngo
  • Computer Science
    Journal of Computer Science and Cybernetics
  • 2021
The cardinality estimation problem -- a corner-stone problem for query optimizers -- is deeply connected to information theoretic inequalities, and inequalities can also be used to derive a couple of classic geometric inequalities such as the Loomis-Whitney inequality.

Deep Sufficient Representation Learning via Mutual Information

A mutual information-based sufficient representation learning (MSRL) approach, which uses the variational formulation of mutual information and leverages the approximation power of deep neural networks, is proposed, which is shown to be consistent in the sense that the conditional probability density function of the response variable given the learned representation converges to the conditional probabilities of the responses given the predictor.

Sufficient Dimension Reduction: An Information-Theoretic Viewpoint

There has been a lot of interest in sufficient dimension reduction (SDR) methodologies, as well as nonlinear extensions in the statistics literature. The SDR methodology has previously been motivated

Entropy and the Brain: An Overview

The present study identifies that the complexity, as quantified by entropy, is a fundamental property of conscious experience, which also plays a vital role in the brain’s capacity for adaptation and therefore whose loss by ageing constitutes a basis for diseases and disorders.

On the Information Plane of Autoencoders

This work derives a theoretical convergence for the IP of autoencoders based on the information-theoretic concept of mutual information (MI), and proposes a new rule to adjust its parameters that compensates scale and dimensionality effects.

Fano's inequality for random variables

Fano's inequality is extended, which controls the average probability of events in terms of the average of some divergences, to work with arbitrary events and even with arbitrary random variables, possibly in continuously infinite number.

Persuasion with Limited Communication Capacity

This work provides an upper bound on the payoffs the persuader can secure by communicating through an imperfect channel that has a fixed and limited number of messages and is subject to exogenous noise and describes this optimal payoff as a function of the information-theoretic capacity of the communication channel.

Multi-destination aggregation with binary symmetric broadcast channel based coding in 802.11 WLANs

Theoretical and simulation results for both unicast and multicast traffic demonstrate that increases in network throughput of more than 100% are possible over a wide range of network conditions and that the much simpler time-sharing scheme yields most of these gains and have minimal loss of performance.

A Cross-Layer Approach to Reducing Packet Delay in Polling-Based Multiuser Systems

This paper establishes that relative to an uncoded system, it is sufficient to reduce the average service time (AST) using FEC to achieve lower average packet delay, and studies and quantifies the reduction in AST that can be achieved using the best possible FEC codes.