Optimal Nonlinear Prediction of Random Fields on Networks

  title={Optimal Nonlinear Prediction of Random Fields on Networks},
  author={Cosma Rohilla Shalizi},
  • C. Shalizi
  • Published in DMCS 12 May 2003
  • Computer Science
It is increasingly common to encounter time-varying random fields on networks (metabolic networks, sensor arrays, distributed computing, etc.).This paper considers the problem of optimal, nonlinear prediction of these fields, showing from an information-theoretic perspective that it is formally identical to the problem of finding minimal local sufficient statistics.I derive general properties of these statistics, show that they can be composed into global predictors, and explore their recursive… 

Figures from this paper

Applications of Computational Mechanics to Modeling Dynamics on Networks

This work model the system as a time-varying random field, and uses various non-parametric statistical methods for inferring dynamical structure from its past dynamics, which allow for the incorporation of both network structure and dynamics into the models.

Quantifying the complexity of random Boolean networks.

  • X. GongJ. Socolar
  • Computer Science
    Physical review. E, Statistical, nonlinear, and soft matter physics
  • 2012
Two measures of the complexity of heterogeneous extended systems are studied, taking random Boolean networks as prototypical cases, and a modification in which complexities of individual nodes are calculated yields vanishing complexity values for networks in the ordered and critical regimes and for highly disordered networks, peaking somewhere in the disordered regime.

Learning spatio-temporal dynamics Nonparametric Methods for Optimal Forecasting and Automated Pattern Discovery

This thesis introduces two new techniques for optimal nonparametric forecasting of spatiotemporal data: hard and mixed LICORS (Light Cone Reconstruction of States).

Inference and Prediction Problems for Spatial and Spatiotemporal Data

This dissertation introduces a novel time series model that improves the accuracy of lung tumor tracking for radiotherapy, and develops a stochastic process model for the spatiotemporal evolution of a basketball possession based on tracking data that records each player’s exact location at 25Hz.

Information processing in two-dimensional cellular automata

It is demonstrated why an accurate model of information processing in two-dimensional cellular automata cannot be constructed from the space-time behavior of these structures, as well as several approaches to automatically identify the spatiotemporal structures with information content.

Estimations of Integrated Information Based on Algorithmic Complexity and Dynamic Querying

It is shown that an object with a high integrated information value is also more compressible, and is, therefore, more sensitive to perturbations, and a perturbation test quantifying compression sensitivity provides a system with a means to extract explanations--causal accounts--of its own behaviour.

Multifield visualization using local statistical complexity

A new approach based on information theoretic concepts is introduced in this paper to detect important regions by extending the concept of local statistical complexity from finite state cellular automata to discretized (multi-)fields.

Based Approach to Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems Permalink

It is illustrated how novel patterns and coherent structures can be discovered in cellular automata and the path from them to climate data is outlined.

A Physics-Based Approach to Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems

A complementary physics-based, data-driven approach that exploits the causal nature of spatiotemporal data sets generated by local dynamics (e.g. hydrodynamic flows) to enhance discovery.

Local Causal States and Discrete Coherent Structures

A formal theory of coherent structures in fully discrete dynamical field theories is presented, builds on the notion of structure introduced by computational mechanics, generalizing it to a local spatiotemporal setting and offers an unsupervised approach to discover and describe coherent structures.



Random Fields on a Network: Modeling, Statistics, and Applications

The theory of spatial models over lattices, or random fields as they are known, has developed significantly over recent years. This book provides a graduate-level introduction to the subject which

Toward a quantitative theory of self-generated complexity

Quantities are defined operationally which qualify as measures of complexity of patterns arising in physical situations, and are essentially Shannon information needed to specify not individual patterns, but either measure-theoretic or algebraic properties of ensembles of pattern arising ina priori translationally invariant situations.

The Structure and Function of Complex Networks

Developments in this field are reviewed, including such concepts as the small-world effect, degree distributions, clustering, network correlations, random graph models, models of network growth and preferential attachment, and dynamical processes taking place on networks.

Graphical Models

Some of the basic ideas underlying graphical models are reviewed, including the algorithmic ideas that allow graphical models to be deployed in large-scale data analysis problems and examples of graphical models in bioinformatics, error-control coding and language processing are presented.

An Algorithm for Pattern Discovery in Time Series

A reliable procedure for building the minimal set of hidden, Markovian states that is statistically capable of producing the behavior exhibited in the data -- the underlying process's causal states.

Computational Mechanics: Pattern and Prediction, Structure and Simplicity

It is shown that the causal-state representation—an ∈-machine—is the minimal one consistent with accurate prediction, and several results are established on ∉-machine optimality and uniqueness and on how∈-machines compare to alternative representations.

Predictive Representations of State

This is the first specific formulation of the predictive idea that includes both stochasticity and actions (controls) and it is shown that any system has a linear predictive state representation with number of predictions no greater than the number of states in its minimal POMDP model.

Quantifying self-organization in cyclic cellular automata

New tools from information theory are introduced that let us calculate the dynamical information content of spatial random processes in CCA, and a complexity measure is introduced to quantitatively determine the rate of self-organization of these cellular automata.

Inferring statistical complexity.

A technique is presented that directly reconstructs minimal equations of motion from the recursive structure of measurement sequences, demonstrating a form of superuniversality that refers only to the entropy and complexity of a data stream.

Complexity of two-dimensional patterns

A complexity measure based on a mean information gain that does not require knowledge of the “maximal” entropy of the pattern, and at the same time sensitively accounts for the inherent correlations in the system.