Informational and Causal Architecture of Continuous-time Renewal Processes

@article{Marzen2017InformationalAC,
  title={Informational and Causal Architecture of Continuous-time Renewal Processes},
  author={Sarah E. Marzen and James P. Crutchfield},
  journal={Journal of Statistical Physics},
  year={2017},
  volume={168},
  pages={109-127}
}
We introduce the minimal maximally predictive models ($$\epsilon \text{-machines }$$ϵ-machines) of processes generated by certain hidden semi-Markov models. Their causal states are either discrete, mixed, or continuous random variables and causal-state transitions are described by partial differential equations. As an application, we present a complete analysis of the $$\epsilon \text{-machines }$$ϵ-machines of continuous-time renewal processes. This leads to closed-form expressions for their… 

Informational and Causal Architecture of Discrete-Time Renewal Processes

TLDR
This work identifies the minimal sufficient statistic for their prediction (the set of causal states), calculates the historical memory capacity required to store those states, delineate what information is predictable (statistical complexity), and decompose the entropy of a single measurement into that shared with the past, future, or both.

Structure and Randomness of Continuous-Time, Discrete-Event Processes

TLDR
The entropy rate and statistical complexity of stochastic processes generated by finite unifilar hidden semi-Markov models—memoryful, state-dependent versions of renewal processes—are calculated for the first time.

Inference, Prediction, and Entropy-Rate Estimation of Continuous-time, Discrete-event Processes

TLDR
The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power and are competitive with the state-of-the-art for prediction and entropy-rate estimation.

Discovering Causal Structure with Reproducing-Kernel Hilbert Space ε-Machines

TLDR
A widely applicable method that infers causal structure directly from observations of a system's behaviors whether they are over discrete or continuous events or time, and robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high-dimensional data.

Superior memory efficiency of quantum devices for the simulation of continuous-time stochastic processes

TLDR
A systematic method for finding superior quantum constructions, and a protocol for analogue simulation of continuous-time renewal processes with a quantum machine that is able to simulate renewal processes—classical stochastic processes that are continuous in time are derived.

Spectral Simplicity of Apparent Complexity, Part I: The Nondiagonalizable Metadynamics of Prediction

TLDR
The first closed-form expressions for complexity measures, couched either in terms of the Drazin inverse or the eigenvalues and projection operators of the appropriate transition dynamic of the recently introduced meromorphic functional calculus are established.

Practical Unitary Simulator for Non-Markovian Complex Processes.

TLDR
A unitary quantum simulator for discrete-time stochastic processes which requires less internal memory than any classical analogue throughout the simulation, and only requires a (small) finite-dimensional Hilbert space.

Intrinsic Computation of a Monod-Wyman-Changeux Molecule

TLDR
This work discusses how statistical complexity changes with slight changes to the underlying model– in this case, a biologically-motivated dynamical model, that of a Monod-Wyman-Changeux molecule.

Memory-efficient tracking of complex temporal and symbolic dynamics with quantum simulators

TLDR
A systematic protocol for quantum model construction relies only on an elementary description of the dynamics of the process, which circumvents restrictions on corresponding classical construction protocols, and allows for a broader range of processes to be modelled efficiently.

The Origins of Computational Mechanics: A Brief Intellectual History and Several Clarifications

TLDR
This brief review clarifies several misunderstandings and addresses concerns recently raised regarding early works in the field (1980s) and shows the significant progress achieved over three decades and points to the many intriguing and outstanding challenges in understanding the computational nature of complex dynamic systems.

References

SHOWING 1-10 OF 37 REFERENCES

Informational and Causal Architecture of Discrete-Time Renewal Processes

TLDR
This work identifies the minimal sufficient statistic for their prediction (the set of causal states), calculates the historical memory capacity required to store those states, delineate what information is predictable (statistical complexity), and decompose the entropy of a single measurement into that shared with the past, future, or both.

Structure and Randomness of Continuous-Time, Discrete-Event Processes

TLDR
The entropy rate and statistical complexity of stochastic processes generated by finite unifilar hidden semi-Markov models—memoryful, state-dependent versions of renewal processes—are calculated for the first time.

Information Symmetries in Irreversible Processes

TLDR
This work examines stationary processes represented or generated by edge-emitting, finite-state hidden Markov models, and shows how to directly calculate a process's fundamental information properties, many of which are otherwise only poorly approximated via process samples.

Anatomy of a Bit: Information in a Time Series Observation

TLDR
The uncertainty of a single observation is dissected to demonstrate how the measures' asymptotic behavior sheds structural and semantic light on the generating process's internal information dynamics.

ON THE ENTROPY FOR SEMI-MARKOV PROCESSES

TLDR
The entropy of the finite distributions of the process is defined, and the entropy rate is obtained explicitly by extending the Shannon-McMillan-Breiman theorem to this class of nonstationary continuous-time processes.

Entropy for semi-Markov processes with Borel state spaces: asymptotic equirepartition properties and invariance principles

The aim of this paper is to define the entropy rate of a semi-Markov process with a Borel state space by extending the strong asymptotic equirepartition property (also called the ergodic theorem of

On entropy for mixtures of discrete and continuous variables

TLDR
The notion of entropy is extended in a natural manner for a mixed-pair random variable, a pair of random variables with one discrete and the other continuous, and leads to an entropy rate for continuous time Markov chains.

Bayesian Structural Inference for Hidden Processes

TLDR
BSI is applied to in-class examples of finite- and infinite-order Markov processes, as well to an out-of-class, infinite-state hidden process, and it is shown that the former more accurately reflects uncertainty in estimated values.