• Corpus ID: 119124856

Nonparametric Maximum Entropy Estimation on Information Diagrams

@article{Martin2016NonparametricME,
  title={Nonparametric Maximum Entropy Estimation on Information Diagrams},
  author={Elliot A. Martin and Jaroslav Hlinka and Alexander Meinke and Filip Dvechtverenko and J{\"o}rn Davidsen},
  journal={arXiv: Data Analysis, Statistics and Probability},
  year={2016}
}
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of random discrete variables when conditioning on bivariate mutual informations and univariate entropies. Specifically, we show how to apply the concept to continuous random variables and vastly expand the types of information-theoretic quantities one can… 

Figures and Tables from this paper

References

SHOWING 1-10 OF 30 REFERENCES

Maximum Entropy Approaches to Living Neural Networks

TLDR
How maximum entropy models have been applied to neuronal ensemble data to account for spatial and temporal correlations is reviewed and criticisms of the maximum entropy approach are discussed that argue that it is not generally applicable to larger ensembles of neurons.

Pairwise network information and nonlinear correlations.

TLDR
This work introduces a novel entropy maximization scheme that is based on conditioning on entropies and mutual informations that is typically superior to other methods based on linear approximations and easier to estimate.

A pairwise maximum entropy model accurately describes resting-state human brain networks

TLDR
It is shown that a pairwise maximum entropy model, which takes into account region-specific activity rates and pairwise interactions, can be robustly and accurately fitted to resting-state human brain activities obtained by functional magnetic resonance imaging and reflects anatomical connexions more accurately than the conventional functional connectivity method.

Weak pairwise correlations imply strongly correlated network states in a neural population

TLDR
It is shown, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons, and it is found that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions.

Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities

TLDR
An asymptotic analysis of the NSB estimator of entropy of a discrete random variable shows that the estimator has a well defined limit for a large cardinality of the studied variable, which allows estimation of entropy with no a priori assumptions about the cardinality.

Inferring the connectivity of coupled oscillators from time-series statistical similarity analysis

TLDR
This work finds that, under adequate conditions, the network links can be perfectly inferred, i.e., no mistakes are made regarding the presence or absence of links.

Statistical mechanics of letters in words.

  • G. StephensW. Bialek
  • Physics
    Physical review. E, Statistical, nonlinear, and soft matter physics
  • 2010
TLDR
This work considers words as a network of interacting letters, and approximate the probability distribution of states taken on by this network, and suggests that these states provide an effective vocabulary which is matched to the frequency of word use and much smaller than the full lexicon.

Network information and connected correlations.

TLDR
The information theoretic analog of connected correlation functions is constructed: irreducible N-point correlation is measured by a decrease in entropy for the joint distribution of N variables relative to the maximum entropy allowed by all the observed N-1 variable distributions.

Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns

TLDR
Analysis of microarray data from genes in Saccharomyces cerevisiae chemostat cultures exhibiting energy metabolic oscillations identifies a gene interaction network that reflects the intracellular communication pathways that adjust cellular metabolic activity and cell division to the limiting nutrient conditions that trigger metabolic oscillation.