Pairwise Maximum Entropy Models for Studying Large Biological Systems: When They Can Work and When They Can't

@article{Roudi2009PairwiseME,
  title={Pairwise Maximum Entropy Models for Studying Large Biological Systems: When They Can Work and When They Can't},
  author={Yasser Roudi and Sheila Nirenberg and Peter E. Latham},
  journal={PLoS Computational Biology},
  year={2009},
  volume={5}
}
One of the most critical problems we face in the study of biological systems is building accurate statistical descriptions of them. This problem has been particularly challenging because biological systems typically contain large numbers of interacting elements, which precludes the use of standard brute force approaches. Recently, though, several groups have reported that there may be an alternate strategy. The reports show that reliable statistical models can be built without knowledge of all… 

Figures from this paper

When are correlations strong
TLDR
An expansion for the entropy of Ising systems in powers of the correlations is developed, carrying this out to fourth order, and it is shown that even though all pairwise correlations are weak, the fact that these correlations are widespread means that their impact on the network as a whole is not captured in the leading orders of perturbation theory.
On the Sufficiency of Pairwise Interactions in Maximum Entropy Models of Networks
TLDR
It is argued that this reduction in complexity can be thought of as a natural property of densely interacting networks in certain regimes, and not necessarily as a special property of living systems.
Statistical Physics of Pairwise Probability Models
TLDR
This paper builds on the previous work on the subject and study the relation between different methods for fitting these models and evaluating their quality, using data from simulated cortical networks to study how the quality of various approximate methods for inferring the parameters in a pairwise model depends on the time bin chosen for binning the data.
On the suciency of pairwise interactions in maximum entropy models of biological networks
TLDR
Based on simulations of random Ising spin networks with p-spin (p > 2) interactions, it is argued that this reduction in complexity can be thought of as a natural property of densely interacting networks in ceramic regimes, and not necessarily as a special property of living systems.
How fast can we learn maximum entropy models of neural populations
TLDR
Using simultaneous recordings of large groups of neurons in the vertebrate retina responding to naturalistic stimuli, it is demonstrated that trading accuracy for entropy may actually improve model performance when data is limited, and an optimization method is suggested that automatically adjusts model constraints in order to achieve good performance.
Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations
TLDR
The results show how only small amounts of randomness are needed to mimic low-order statistical properties of highly entropic distributions, and some applications for engineered and biological information transmission systems are discussed.
Searching for Collective Behavior in a Large Network of Sensory Neurons
TLDR
The properties of the neural vocabulary are explored by estimating its entropy, which constrains the population's capacity to represent visual information, and classifying activity patterns into a small set of metastable collective modes, showing that the neural codeword ensembles are extremely inhomogenous.
When are microcircuits well-modeled by maximum entropy methods?
TLDR
Overall, the findings identify circuit-level mechanisms that produce and fail to produce higher-order spiking statistics in neural ensembles.
...
...

References

SHOWING 1-10 OF 40 REFERENCES
Weak pairwise correlations imply strongly correlated network states in a neural population
TLDR
It is shown, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons, and it is found that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions.
Near-Maximum Entropy Models for Binary Neural Representations of Natural Images
TLDR
The NearMaxEnt approach can serve as a tool for testing predictions from a pairwise maximum entropy model not only for low-dimensional marginals, but also for high dimensional measurements of more than thousand units, and is demonstrated by studying natural images with dichotomized pixel intensities.
Multivariate Information Bottleneck
TLDR
A general principled framework for multivariate extensions of the information bottleneck method is introduced that provides insights about bottleneck variations and enables us to characterize the solutions of these variations.
A Small World of Neuronal Synchrony
A small-world network has been suggested to be an efficient solution for achieving both modular and global processing—a property highly desirable for brain computations. Here, we investigated
A Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical Networks In Vitro
TLDR
Although a second-order maximum entropy model successfully predicts correlated states in cortical networks, it should be extended to account for temporal correlations observed between states, and a significant relationship between strong pairwise temporal correlations and observed sequence length is found.
The Structure of Multi-Neuron Firing Patterns in Primate Retina
TLDR
Large-scale multi-electrode recordings were used to measure electrical activity in nearly complete, regularly spaced mosaics of several hundred ON and OFF parasol retinal ganglion cells in macaque monkey retina, and pairwise and adjacent interactions accurately accounted for the structure and prevalence of multi-neuron firing patterns.
Inferring the capacity of the vector Poisson channel with a Bernoulli model
TLDR
By modeling multi-neuron recordings as a sum of a neural population, it is shown that the resulting capacity is much less than the population's, reducing it to a level that can be less than provided with two separated neural responses.
Theory for the folding and stability of globular proteins.
  • K. Dill
  • Biology, Chemistry
    Biochemistry
  • 1985
TLDR
Using lattice statistical mechanics, theory is developed to account for the folding of a heteropolymer molecule such as a protein to the globular and soluble state and the number of accessible conformations is calculated to be an exceedingly small fraction of the number available to the random coil.
Natural-like function in artificial WW domains
TLDR
Construction of artificial protein sequences directed only by the SCA showed that the information extracted by this analysis is sufficient to engineer the WW fold at atomic resolution, and it was demonstrated that these artificial WW sequences function like their natural counterparts, showing class-specific recognition of proline-containing target peptides.
The Structure of Large-Scale Synchronized Firing in Primate Retina
TLDR
Large-scale synchronized firing in the entire population of ON-parasol cells appears to reflect simple neighbor interactions, rather than a unique visual signal or a highly redundant coding scheme.
...
...