Statistical Physics of Pairwise Probability Models

@article{Roudi2009StatisticalPO,
  title={Statistical Physics of Pairwise Probability Models},
  author={Yasser Roudi and Erik Aurell and John A. Hertz},
  journal={Frontiers in Computational Neuroscience},
  year={2009},
  volume={3}
}
Statistical models for describing the probability distribution over the states of biological systems are commonly used for dimensional reduction. Among these models, pairwise models are very attractive in part because they can be fit using a reasonable amount of data: knowledge of the mean values and correlations between pairs of elements in the system is sufficient. Not surprisingly, then, using pairwise models for studying neural data has been the focus of many studies in recent years. In… 

Figures from this paper

Estimation Bias in Maximum Entropy Models
Maximum entropy models have become popular statistical models in neuroscience and other areas in biology and can be useful tools for obtaining estimates of mutual information in biological systems.
Inverse inference in the asymmetric Ising model
TLDR
This thesis is to study statistical methods for inferring information about the connectivity of complex networks starting from experimental data from a statistical physics point of view drawing from the arsenal of methods developed in the study of spin glasses.
How biased are maximum entropy models?
TLDR
This work provides a perturbative approximation of the maximally expected bias when the true model is out of model class, and illustrates the results using numerical simulations of an Ising model; i.e. the second-order maximum entropy distribution on binary data.
Approximate Inference for Time-Varying Interactions and Macroscopic Dynamics of Neural Populations
TLDR
By introducing multiple analytic approximation methods to a state-space model of neural population activity, this work makes it possible to estimate dynamic pairwise interactions of up to 60 neurons and accurately estimates dynamics of network properties such as sparseness, entropy, and heat capacity by simulated data.
On the criticality of inferred models
TLDR
This work argues that inference procedures are likely to yield models which are close to singular values of parameters, akin to critical points in physics where phase transitions occur, and discusses inference of interacting point processes with application to financial data, and shows that sensible choices of observation time scales naturally yield modelsWhich areclose to criticality.
On the typical properties of inverse problems in statistical mechanics
TLDR
This work considers the problem of extracting a set of interaction parameters from an high-dimensional dataset describing T independent configurations of a complex system composed of N binary units and presents a class of models for which the analytical solution of this inverse problem can be obtained.
Edinburgh Research Explorer How biased are maximum entropy models?
TLDR
This work provides a perturbative approximation of the maximally expected bias when the true model is out of model class, and illustrates the results using numerical simulations of an Ising model; i.e. the second-order maximum entropy distribution on binary data.
Spin glass models for a network of real neurons
TLDR
It is shown that Pairwise interactions between neurons account for observed higher-order correlations, and that for groups of 10 or more neurons pairwise interactions can no longer be regarded as small perturbations in an independent system.
Searching for Collective Behavior in a Large Network of Sensory Neurons
TLDR
The properties of the neural vocabulary are explored by estimating its entropy, which constrains the population's capacity to represent visual information, and classifying activity patterns into a small set of metastable collective modes, showing that the neural codeword ensembles are extremely inhomogenous.
Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models
TLDR
This work reviews undirected pairwise maximum-entropy probability models in two categories of data types, those with continuous and categorical random variables, and presents recently developed inference methods from the field of protein contact prediction.
...
...

References

SHOWING 1-10 OF 46 REFERENCES
Ising model for neural data: model quality and approximate methods for extracting functional connectivity.
TLDR
P pairwise Ising models for describing the statistics of multineuron spike trains are studied, using data from a simulated cortical network and the quality of these models is studied by comparing their entropies with that of the data, finding that they perform well for small subsets of the neurons in the network, but the fit quality starts to deteriorate as the subset size grows.
Pairwise Maximum Entropy Models for Studying Large Biological Systems: When They Can Work and When They Can't
TLDR
This work provides a general framework for determining the extent to which pairwise models can be used to predict the behavior of large biological systems and shows that, in most cases, they will not.
Faster solutions of the inverse pairwise Ising problem
TLDR
A combination of recent coordinate descent algorithms with an adaptation of the histogram MonteCarlo method is used, and the resulting algorithm learns the parameters of an Ising model describing a network of forty neurons within a few minutes.
Theory of correlations in stochastic neural networks.
  • Ginzburg, Sompolinsky
  • Computer Science
    Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics
  • 1994
TLDR
The theory of neuronal correlation functions in large networks comprising of several highly connected subpopulations, and obeying stochastic dynamic rules is developed and extended to networks with random connectivity, such as randomly dilute networks.
Weak pairwise correlations imply strongly correlated network states in a neural population
TLDR
It is shown, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons, and it is found that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions.
Analyzing Functional Connectivity Using a Network Likelihood Model of Ensemble Neural Spiking Activity
TLDR
A discrete-time version of Chornoboy, Schramm, and Karr's maximum likelihood method for the simultaneous analysis of multiple pair-wise interactions among an ensemble of neurons is devised that includes a new, efficient computational strategy, a principled method to compute starting values, and a principled stopping criterion.
Ising models for networks of real neurons
TLDR
Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins, and here they are constructed that describe the correlated spiking activity of populations of 40 neurons in the retina.
Near-Maximum Entropy Models for Binary Neural Representations of Natural Images
TLDR
The NearMaxEnt approach can serve as a tool for testing predictions from a pairwise maximum entropy model not only for low-dimensional marginals, but also for high dimensional measurements of more than thousand units, and is demonstrated by studying natural images with dichotomized pixel intensities.
A Small World of Neuronal Synchrony
A small-world network has been suggested to be an efficient solution for achieving both modular and global processing—a property highly desirable for brain computations. Here, we investigated
...
...