Effective Complexity of Stationary Process Realizations

@article{Ay2011EffectiveCO,
  title={Effective Complexity of Stationary Process Realizations},
  author={Nihat Ay and Markus M{\"u}ller and Arleta Szkola},
  journal={Entropy},
  year={2011},
  volume={13},
  pages={1200-1211}
}
Abstract: The concept of effective complexity of an object as the minimal descriptionlength of its regularities has been initiated by Gell-Mann and Lloyd. The regularities aremodeled by means of ensembles, which is the probability distributions on finite binarystrings. In our previous paper [1] we propose a definition of effective complexity in preciseterms of algorithmic information theory. Here we investigate the effective complexity ofbinary strings generated by stationary, in general not… 

Effective Complexity and Its Relation to Logical Depth

TLDR
A remarkable relation between effective complexity and Bennett's logical depth is shown: If the effective complexity of a string x exceeds a certain explicit threshold then that string must have astronomically large depth; otherwise, the depth can be arbitrarily small.

Some comments on computational mechanics, complexity measures, and all that

TLDR
It is shown that there exist simple models where none of the nodes of the "$\epsilon$-machine" (the "causal states") corresponds to an element of a state (or history) space partition, and a simple and precise way of estimating excess entropy is presented.

Fields of Application of Information Geometry

1. Complexity measures can be geometrically built by using the information distance (Kullback–Leibler divergence) from families with restricted statistical dependencies. The Pythagorean geometry

A geometric approach to complexity.

TLDR
A geometric approach to complexity based on the principle that complexity requires interactions at different scales of description is developed, which presents a theory of complexity measures for finite random fields using the geometric framework of hierarchies of exponential families.

Abstraction in Artificial Intelligence and Complex Systems

TLDR
A formal model, the KRAmodel, is presented to capture the characterizing properties of abstraction, and its pragmatic power is illustrated with three case studies: Model-based diagnosis, Cartographic Generalization, and learning Hierarchical Hidden Markov Models.

References

SHOWING 1-10 OF 22 REFERENCES

Effective Complexity and Its Relation to Logical Depth

TLDR
A remarkable relation between effective complexity and Bennett's logical depth is shown: If the effective complexity of a string x exceeds a certain explicit threshold then that string must have astronomically large depth; otherwise, the depth can be arbitrarily small.

Effective Complexity

TLDR
The effective complexity (EC) of an entity is defined as the length of a highly compressed description of its regularities, and a formal approach is needed both to the notion of minimum description length and to the distinction between regularities and those features that are treated as random or incidental.

Kolmogorov's structure functions and model selection

TLDR
The goodness-of-fit of an individual model with respect to individual data is precisely quantify and it is shown that-within the obvious constraints-every graph is realized by the structure function of some data.

Meaningful Information

  • P. Vitányi
  • Computer Science, Mathematics
    IEEE Transactions on Information Theory
  • 2006
TLDR
The theory of recursive functions statistic, the maximum and minimum value, the existence of absolutely nonstochastic objects (that have maximal sophistication-all the information in them is meaningful and there is no residual randomness), and the relation to the halting problem and further algorithmic properties are developed.

Information measures, effective complexity, and total information

This article defines the concept of an information measure and shows how common information measures such as entropy, Shannon information, and algorithmic information content can be combined to solve

Sophistication Revisited

TLDR
This work formalizes a connection between sophistication and a variation of computational depth and proves the existence of strings with maximum sophistication and shows that they are the deepest of all strings.

An Introduction to Kolmogorov Complexity and Its Applications

TLDR
The book presents a thorough treatment of the central ideas and their applications of Kolmogorov complexity with a wide range of illustrative applications, and will be ideal for advanced undergraduate students, graduate students, and researchers in computer science, mathematics, cognitive sciences, philosophy, artificial intelligence, statistics, and physics.

Logical depth and physical complexity

Some mathematical and natural objects (a random sequence, a sequence of zeros, a perfect crystal, a gas) are intuitively trivial, while others (e.g. the human body, the digits of π) contain internal

Algorithmic randomness and physical entropy.

  • Zurek
  • Computer Science
    Physical review. A, General physics
  • 1989
TLDR
Algorithmic randomness is typically very difficult to calculate exactly but relatively easy to estimate, and is essential in describing the operation of thermodynamic engines from the viewpoint of information gathering and using systems.

Book Review: An introduction to Kolmogorov Complexity and its Applications Second Edition, 1997 by Ming Li and Paul Vitanyi (Springer (Graduate Text Series))

TLDR
The complexity of a string is defined as the shortest description of x, and a formal definition is given that is equivalent to the one in the book.