Optimal Nonlinear Prediction of Random Fields on Networks

  title={Optimal Nonlinear Prediction of Random Fields on Networks},
  author={Cosma Rohilla Shalizi},
  • C. Shalizi
  • Published in DMCS 12 May 2003
  • Computer Science
It is increasingly common to encounter time-varying random fields on networks (metabolic networks, sensor arrays, distributed computing, etc.).This paper considers the problem of optimal, nonlinear prediction of these fields, showing from an information-theoretic perspective that it is formally identical to the problem of finding minimal local sufficient statistics.I derive general properties of these statistics, show that they can be composed into global predictors, and explore their recursive… 

Figures from this paper

Applications of Computational Mechanics to Modeling Dynamics on Networks

This work model the system as a time-varying random field, and uses various non-parametric statistical methods for inferring dynamical structure from its past dynamics, which allow for the incorporation of both network structure and dynamics into the models.

Quantifying the complexity of random Boolean networks.

  • X. GongJ. Socolar
  • Computer Science
    Physical review. E, Statistical, nonlinear, and soft matter physics
  • 2012
Two measures of the complexity of heterogeneous extended systems are studied, taking random Boolean networks as prototypical cases, and a modification in which complexities of individual nodes are calculated yields vanishing complexity values for networks in the ordered and critical regimes and for highly disordered networks, peaking somewhere in the disordered regime.

Balance between Noise and Information Flow Maximizes Set Complexity of Network Dynamics

The results suggest that the maximization of complexity near to the state transition might be a more general phenomenon in physical systems, and that noise present in a system may in fact be useful in retaining the system in a state with high information content.

Learning spatio-temporal dynamics Nonparametric Methods for Optimal Forecasting and Automated Pattern Discovery

This thesis introduces two new techniques for optimal nonparametric forecasting of spatiotemporal data: hard and mixed LICORS (Light Cone Reconstruction of States).

Inference and Prediction Problems for Spatial and Spatiotemporal Data

This dissertation introduces a novel time series model that improves the accuracy of lung tumor tracking for radiotherapy, and develops a stochastic process model for the spatiotemporal evolution of a basketball possession based on tracking data that records each player’s exact location at 25Hz.

Information processing in two-dimensional cellular automata

It is demonstrated why an accurate model of information processing in two-dimensional cellular automata cannot be constructed from the space-time behavior of these structures, as well as several approaches to automatically identify the spatiotemporal structures with information content.

Estimations of Integrated Information Based on Algorithmic Complexity and Dynamic Querying

It is shown that an object with a high integrated information value is also more compressible, and is, therefore, more sensitive to perturbations, and a perturbation test quantifying compression sensitivity provides a system with a means to extract explanations--causal accounts--of its own behaviour.

Mixed LICORS: A Nonparametric Algorithm for Predictive State Reconstruction

Mixed LICORS extends the recent LICORS algorithm from hard clustering of predictive distributions to a non-parametric, EM-like soft clustering, which retains the asymptotic predictive optimality of LICORS, but greatly improves out-of-sample forecasts with limited data.

Multifield visualization using local statistical complexity

A new approach based on information theoretic concepts is introduced in this paper to detect important regions by extending the concept of local statistical complexity from finite state cellular automata to discretized (multi-)fields.

Based Approach to Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems Permalink

It is illustrated how novel patterns and coherent structures can be discovered in cellular automata and the path from them to climate data is outlined.



Random Fields on a Network: Modeling, Statistics, and Applications

The theory of spatial models over lattices, or random fields as they are known, has developed significantly over recent years. This book provides a graduate-level introduction to the subject which

Toward a quantitative theory of self-generated complexity

Quantities are defined operationally which qualify as measures of complexity of patterns arising in physical situations, and are essentially Shannon information needed to specify not individual patterns, but either measure-theoretic or algebraic properties of ensembles of pattern arising ina priori translationally invariant situations.

The Structure and Function of Complex Networks

Developments in this field are reviewed, including such concepts as the small-world effect, degree distributions, clustering, network correlations, random graph models, models of network growth and preferential attachment, and dynamical processes taking place on networks.

Graphical Models

Some of the basic ideas underlying graphical models are reviewed, including the algorithmic ideas that allow graphical models to be deployed in large-scale data analysis problems and examples of graphical models in bioinformatics, error-control coding and language processing are presented.

An Algorithm for Pattern Discovery in Time Series

A reliable procedure for building the minimal set of hidden, Markovian states that is statistically capable of producing the behavior exhibited in the data -- the underlying process's causal states.

Computational Mechanics: Pattern and Prediction, Structure and Simplicity

It is shown that the causal-state representation—an ∈-machine—is the minimal one consistent with accurate prediction, and several results are established on ∉-machine optimality and uniqueness and on how∈-machines compare to alternative representations.

Predictive Representations of State

This is the first specific formulation of the predictive idea that includes both stochasticity and actions (controls) and it is shown that any system has a linear predictive state representation with number of predictions no greater than the number of states in its minimal POMDP model.

Quantifying self-organization in cyclic cellular automata

New tools from information theory are introduced that let us calculate the dynamical information content of spatial random processes in CCA, and a complexity measure is introduced to quantitatively determine the rate of self-organization of these cellular automata.

Inferring statistical complexity.

A technique is presented that directly reconstructs minimal equations of motion from the recursive structure of measurement sequences, demonstrating a form of superuniversality that refers only to the entropy and complexity of a data stream.

Complexity of two-dimensional patterns

A complexity measure based on a mean information gain that does not require knowledge of the “maximal” entropy of the pattern, and at the same time sensitively accounts for the inherent correlations in the system.