Coherent information structure in complex computation

@article{Lizier2011CoherentIS,
  title={Coherent information structure in complex computation},
  author={Joseph T. Lizier and Mikhail Prokopenko and Albert Y. Zomaya},
  journal={Theory in Biosciences},
  year={2011},
  volume={131},
  pages={193-203}
}
We have recently presented a framework for the information dynamics of distributed computation that locally identifies the component operations of information storage, transfer, and modification. We have observed that while these component operations exist to some extent in all types of computation, complex computation is distinguished in having coherent structure in its local information dynamics profiles. In this article, we conjecture that coherent information structure is a defining feature… 
Local measures of information storage in complex distributed computation
Information modification and particle collisions in distributed computation.
TLDR
The separable information is introduced, a measure which locally identifies information modification events where separate inspection of the sources to a computation is misleading about its outcome.
Measuring the Dynamics of Information Processing on a Local Scale in Time and Space
TLDR
This chapter reviews the mathematics of how to measure local entropy and mutual information values at specific observations of time-series processes, and describes how these measures can reveal much more intricate details about the dynamics of complex systems than their more well-known “average” measures do.
Entropy balance and information processing in bipartite and nonbipartite composite systems
TLDR
This paper constructs irreversibility measures in terms of extant information theoretic measures and relates them to the physical entropy productions that characterise the behaviour of single and composite systems in stochastic thermodynamics illustrating them with simple examples.
Bits from Biology for Computational Intelligence
TLDR
This article shows how to analyze neural systems using information theory to obtain constraints that help identify the algorithms run by such systems and the information they represent.
Computation in Complex Systems
TLDR
This approach seeks a fundamental understanding of how collective behaviour results from these interactions between simple individuals, examining both natural and man-made systems as apparently diverse as insect colonies, the brain, the immune system, economies and the world wide web.
Information processing in echo state networks at the edge of chaos
TLDR
Evidence is presented that both information transfer and storage in the recurrent layer are maximized close to this phase transition, providing an explanation for why guiding the recurrent layers toward the edge of chaos is computationally useful.
On the Effect of Heterogeneity on the Dynamics and Performance of Dynamical Networks
TLDR
This thesis aims to study the effect of heterogeneity in the dynamics of physical systems that pertains to information processing and introduces the model parameters to characterize the heterogeneity of components in RBN and LSM networks.
Bits from Brains for Biologically Inspired Computing
TLDR
This article shows how to analyze neural systems using information theory with the aim of obtaining constraints that help to identify the algorithms run by neural systems and the information they represent.
...
...

References

SHOWING 1-10 OF 57 REFERENCES
The local information dynamics of distributed computation in complex systems
TLDR
A complete information-theoretic framework to quantify these operations on information, and in particular their dynamics in space and time, is applied to cellular automata, and delivers important insights into the fundamental nature of distributed computation and the dynamics of complex systems.
Local measures of information storage in complex distributed computation
The organization of intrinsic computation: complexity-entropy diagrams and the diversity of natural information processing.
TLDR
This work uses complexity-entropy diagrams to analyze intrinsic computation in a broad array of deterministic nonlinear and linear stochastic processes, including maps of the interval, cellular automata, and Ising spin systems in one and two dimensions, Markov chains, and probabilistic minimal finite-state machines.
Computation at the edge of chaos: Phase transitions and emergent computation
Information modification and particle collisions in distributed computation.
TLDR
The separable information is introduced, a measure which locally identifies information modification events where separate inspection of the sources to a computation is misleading about its outcome.
Automatic filters for the detection of coherent structure in spatiotemporal systems.
TLDR
Two approaches to automatically filter the changing configurations of spatial dynamical systems and extract coherent structures are presented, a modification of the local Lyapunov exponent approach suitable to cellular automata and other discrete spatial systems and a more traditional approach based on an order parameter and free energy is compared.
Local information transfer as a spatiotemporal filter for complex systems.
TLDR
A measure of local information transfer, derived from an existing averaged information-theoretical measure, namely, transfer entropy, is presented, providing the first quantitative evidence for the long-held conjecture that the emergent traveling coherent structures known as particles are the dominant information transfer agents in cellular automata.
Detecting Non-trivial Computation in Complex Dynamics
TLDR
This work quantifies the local information dynamics at each spatiotemporal point in a complex system in terms of each element of computation: information storage, transfer and modification to provide the first quantitative evidence that collisions between particles therein are the dominant information modification events.
The Information Dynamics of Phase Transitions in Random Boolean Networks
TLDR
This work uses a recently published framework to characterize the distributed computation in terms of its underlying information dynamics: information storage, information transfer and information modification, and finds maximizations in information storage and coherent information transfer on either side of the critical point.
Information Theory of Complex Networks: On Evolution and Architectural Constraints
TLDR
It is shown that real networks are clustered in a well-defined domain of the entropy- noise space and that optimally heterogeneous nets actually cluster around the same narrow domain, suggesting that strong constraints actually operate on the possible universe of complex networks.
...
...