Statistical Physics of Pairwise Probability Models

  title={Statistical Physics of Pairwise Probability Models},
  author={Yasser Roudi and Erik Aurell and John A. Hertz},
  journal={Frontiers in Computational Neuroscience},
Statistical models for describing the probability distribution over the states of biological systems are commonly used for dimensional reduction. Among these models, pairwise models are very attractive in part because they can be fit using a reasonable amount of data: knowledge of the mean values and correlations between pairs of elements in the system is sufficient. Not surprisingly, then, using pairwise models for studying neural data has been the focus of many studies in recent years. In… 

Figures from this paper

A statistical physics approach to inference problems on random networks
A novel mean-field solution to the discrete time parallel dynamics of the Ising model is derived, based on a weak coupling expansion of the log-generating function with constrained first and second order moments over time, the result of which outperforms other mean field techniques in predicting single site magnetization.
Estimation Bias in Maximum Entropy Models
Maximum entropy models have become popular statistical models in neuroscience and other areas in biology and can be useful tools for obtaining estimates of mutual information in biological systems.
Inverse inference in the asymmetric Ising model
This thesis is to study statistical methods for inferring information about the connectivity of complex networks starting from experimental data from a statistical physics point of view drawing from the arsenal of methods developed in the study of spin glasses.
How biased are maximum entropy models?
This work provides a perturbative approximation of the maximally expected bias when the true model is out of model class, and illustrates the results using numerical simulations of an Ising model; i.e. the second-order maximum entropy distribution on binary data.
Approximate Inference for Time-Varying Interactions and Macroscopic Dynamics of Neural Populations
By introducing multiple analytic approximation methods to a state-space model of neural population activity, this work makes it possible to estimate dynamic pairwise interactions of up to 60 neurons and accurately estimates dynamics of network properties such as sparseness, entropy, and heat capacity by simulated data.
A statistical physics approach to learning curves for the inverse Ising problem
Using methods of statistical physics, we analyse the error of learning couplings in large Ising models from independent data (the inverse Ising problem). We concentrate on learning based on local
On the typical properties of inverse problems in statistical mechanics
This work considers the problem of extracting a set of interaction parameters from an high-dimensional dataset describing T independent configurations of a complex system composed of N binary units and presents a class of models for which the analytical solution of this inverse problem can be obtained.
Edinburgh Research Explorer How biased are maximum entropy models?
This work provides a perturbative approximation of the maximally expected bias when the true model is out of model class, and illustrates the results using numerical simulations of an Ising model; i.e. the second-order maximum entropy distribution on binary data.
Spin glass models for a network of real neurons
It is shown that Pairwise interactions between neurons account for observed higher-order correlations, and that for groups of 10 or more neurons pairwise interactions can no longer be regarded as small perturbations in an independent system.
Searching for Collective Behavior in a Large Network of Sensory Neurons
The properties of the neural vocabulary are explored by estimating its entropy, which constrains the population's capacity to represent visual information, and classifying activity patterns into a small set of metastable collective modes, showing that the neural codeword ensembles are extremely inhomogenous.


Ising model for neural data: model quality and approximate methods for extracting functional connectivity.
P pairwise Ising models for describing the statistics of multineuron spike trains are studied, using data from a simulated cortical network and the quality of these models is studied by comparing their entropies with that of the data, finding that they perform well for small subsets of the neurons in the network, but the fit quality starts to deteriorate as the subset size grows.
Pairwise Maximum Entropy Models for Studying Large Biological Systems: When They Can Work and When They Can't
This work provides a general framework for determining the extent to which pairwise models can be used to predict the behavior of large biological systems and shows that, in most cases, they will not.
Faster solutions of the inverse pairwise Ising problem
A combination of recent coordinate descent algorithms with an adaptation of the histogram MonteCarlo method is used, and the resulting algorithm learns the parameters of an Ising model describing a network of forty neurons within a few minutes.
Theory of correlations in stochastic neural networks.
  • Ginzburg, Sompolinsky
  • Computer Science
    Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics
  • 1994
The theory of neuronal correlation functions in large networks comprising of several highly connected subpopulations, and obeying stochastic dynamic rules is developed and extended to networks with random connectivity, such as randomly dilute networks.
Weak pairwise correlations imply strongly correlated network states in a neural population
It is shown, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons, and it is found that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions.
Analyzing Functional Connectivity Using a Network Likelihood Model of Ensemble Neural Spiking Activity
A discrete-time version of Chornoboy, Schramm, and Karr's maximum likelihood method for the simultaneous analysis of multiple pair-wise interactions among an ensemble of neurons is devised that includes a new, efficient computational strategy, a principled method to compute starting values, and a principled stopping criterion.
Ising models for networks of real neurons
Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins, and here they are constructed that describe the correlated spiking activity of populations of 40 neurons in the retina.
Near-Maximum Entropy Models for Binary Neural Representations of Natural Images
The NearMaxEnt approach can serve as a tool for testing predictions from a pairwise maximum entropy model not only for low-dimensional marginals, but also for high dimensional measurements of more than thousand units, and is demonstrated by studying natural images with dichotomized pixel intensities.
A Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical Networks In Vitro
Although a second-order maximum entropy model successfully predicts correlated states in cortical networks, it should be extended to account for temporal correlations observed between states, and a significant relationship between strong pairwise temporal correlations and observed sequence length is found.
A Small World of Neuronal Synchrony
A small-world network has been suggested to be an efficient solution for achieving both modular and global processing—a property highly desirable for brain computations. Here, we investigated