Effective Complexity of Stationary Process Realizations

@article{Ay2010EffectiveCO,
  title={Effective Complexity of Stationary Process Realizations},
  author={Nihat Ay and Markus M{\"u}ller and Arleta Szkola},
  journal={Entropy},
  year={2010},
  volume={13},
  pages={1200-1211}
}
Abstract: The concept of effective complexity of an object as the minimal descriptionlength of its regularities has been initiated by Gell-Mann and Lloyd. The regularities aremodeled by means of ensembles, which is the probability distributions on finite binarystrings. In our previous paper [1] we propose a definition of effective complexity in preciseterms of algorithmic information theory. Here we investigate the effective complexity ofbinary strings generated by stationary, in general not… 

Effective Complexity and Its Relation to Logical Depth

A remarkable relation between effective complexity and Bennett's logical depth is shown: If the effective complexity of a string x exceeds a certain explicit threshold then that string must have astronomically large depth; otherwise, the depth can be arbitrarily small.

Fields of Application of Information Geometry

1. Complexity measures can be geometrically built by using the information distance (Kullback–Leibler divergence) from families with restricted statistical dependencies. The Pythagorean geometry

A geometric approach to complexity.

A geometric approach to complexity based on the principle that complexity requires interactions at different scales of description is developed, which presents a theory of complexity measures for finite random fields using the geometric framework of hierarchies of exponential families.

Abstraction in Artificial Intelligence and Complex Systems

A formal model, the KRAmodel, is presented to capture the characterizing properties of abstraction, and its pragmatic power is illustrated with three case studies: Model-based diagnosis, Cartographic Generalization, and learning Hierarchical Hidden Markov Models.

Some comments on computational mechanics, complexity measures, and all that

It is shown that there exist simple models where none of the nodes of the "$\epsilon$-machine" (the "causal states") corresponds to an element of a state (or history) space partition, and a simple and precise way of estimating excess entropy is presented.

References

SHOWING 1-10 OF 22 REFERENCES

Effective Complexity and Its Relation to Logical Depth

A remarkable relation between effective complexity and Bennett's logical depth is shown: If the effective complexity of a string x exceeds a certain explicit threshold then that string must have astronomically large depth; otherwise, the depth can be arbitrarily small.

Effective Complexity

The effective complexity (EC) of an entity is defined as the length of a highly compressed description of its regularities, and a formal approach is needed both to the notion of minimum description length and to the distinction between regularities and those features that are treated as random or incidental.

Kolmogorov's structure functions and model selection

The goodness-of-fit of an individual model with respect to individual data is precisely quantify and it is shown that-within the obvious constraints-every graph is realized by the structure function of some data.

Meaningful Information

  • P. Vitányi
  • Computer Science, Mathematics
    IEEE Transactions on Information Theory
  • 2006
The theory of recursive functions statistic, the maximum and minimum value, the existence of absolutely nonstochastic objects (that have maximal sophistication-all the information in them is meaningful and there is no residual randomness), and the relation to the halting problem and further algorithmic properties are developed.

Sophistication Revisited

This work formalizes a connection between sophistication and a variation of computational depth and proves the existence of strings with maximum sophistication and shows that they are the deepest of all strings.

An Introduction to Kolmogorov Complexity and Its Applications

The book presents a thorough treatment of the central ideas and their applications of Kolmogorov complexity with a wide range of illustrative applications, and will be ideal for advanced undergraduate students, graduate students, and researchers in computer science, mathematics, cognitive sciences, philosophy, artificial intelligence, statistics, and physics.

Logical depth and physical complexity

Some mathematical and natural objects (a random sequence, a sequence of zeros, a perfect crystal, a gas) are intuitively trivial, while others (e.g. the human body, the digits of π) contain internal

Algorithmic randomness and physical entropy.

  • Zurek
  • Computer Science
    Physical review. A, General physics
  • 1989
Algorithmic randomness is typically very difficult to calculate exactly but relatively easy to estimate, and is essential in describing the operation of thermodynamic engines from the viewpoint of information gathering and using systems.

Book Review: An introduction to Kolmogorov Complexity and its Applications Second Edition, 1997 by Ming Li and Paul Vitanyi (Springer (Graduate Text Series))

The complexity of a string is defined as the shortest description of x, and a formal definition is given that is equivalent to the one in the book.

Algorithmic statistics

The algorithmic theory of statistic, sufficient statistic, and minimal sufficient statistic is developed and it is shown that a function is a probabilistic sufficient statistic iff it is with high probability (in an appropriate sense) an algorithmic sufficient statistic.