# Complexity-based induction systems: Comparisons and convergence theorems

@article{Solomonoff1978ComplexitybasedIS, title={Complexity-based induction systems: Comparisons and convergence theorems}, author={Ray J. Solomonoff}, journal={IEEE Trans. Inf. Theory}, year={1978}, volume={24}, pages={422-432} }

In 1964 the author proposed as an explication of {\em a priori} probability the probability measure induced on output strings by a universal Turing machine with unidirectional output tape and a randomly coded unidirectional input tape. Levin has shown that if tilde{P}'_{M}(x) is an unnormalized form of this measure, and P(x) is any computable probability measure on strings, x , then \tilde{P}'_{M}\geqCP(x) where C is a constant independent of x . The corresponding result for the normalized form…

## 398 Citations

Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit

- Mathematics, Computer ScienceInt. J. Found. Comput. Sci.
- 2002

A natural hierarchy of generalizations of algorithmic probability and Kolmogorov complexity is obtained, suggesting that the "true" information content of some bitstring x is the size of the shortest nonhalting program that converges to x and nothing but x on a Turing machine that can edit its previous outputs.

The Semimeasure Property of Algorithmic Probability - "Feature" or "Bug"?

- Computer ScienceAlgorithmic Probability and Friends
- 2011

This paper argues that the semimeasure property contributes substantially, in its own right, to the power of an algorithmic-probability-based theory of induction, and that normalization is unnecessary.

KOLMOGOROV'S CONTRIBUTIONS TO INFORMATION THEORY AND ALGORITHMIC COMPLEXITY

- Mathematics, Computer Science
- 1989

If the authors let Pu(x) = Pr{U prints x} be the probability that a given computer U prints x when given a random program, it can be shown that log(1/Pu(x)) - K( x) for all x, thus establishing a vital link between the "universal" probability measure Pu and the " universal" complexity K.

Coding-theorem like behaviour and emergence of the universal distribution from resource-bounded algorithmic probability

- Computer ScienceInt. J. Parallel Emergent Distributed Syst.
- 2019

It is shown that up to 60% of the simplicity/complexity bias in distributions produced even by the weakest of the computational models can be accounted for by Algorithmic Probability in its approximation to the Universal Distribution.

The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions

- Mathematics, Computer ScienceCOLT
- 2002

This work replaces Solomonoff's optimal but noncomputable method for inductive inference with the novel Speed Prior S, under which the cumulative a priori probability of all data whose computation through an optimal algorithm requires more than O(n) resources is 1/n.

Inductive Reasoning and Kolmogorov Complexity

- Computer ScienceJ. Comput. Syst. Sci.
- 1992

The thesis is developed that Solomonoff's method is fundamental in the sense that many other induction principles can be viewed as particular ways to obtain computable approximations to it.

Predictions and algorithmic statistics for infinite sequence

- Mathematics, Computer ScienceArXiv
- 2020

A new way for prediction is suggested that for every finite string $x$ the authors predict the new bit according to the best (in some sence) distribution for £x and this method has no that negative aspect as Solomonoff's method.

Merging with a set of probability measures: A characterization

- Economics, Computer Science
- 2015

It is argued that the characterization result can be extended to the case of infinitely repeated games and has some interesting applications with regard to the impossibility result in Nachbar (1997, 2005).

The Kolmogorov Lecture* The Universal Distribution and Machine Learning

- Computer ScienceComput. J.
- 2003

This lecture discusses the Universal Distribution and some of its properties: its accuracy, its incomputability, its subjectivity, and how to use this distribution to create very intelligent machines.

Convergence and Error Bounds for Universal Prediction of Nonbinary Sequences

- Computer ScienceECML
- 2001

It is shown that the average number of prediction errors made by the universal ξ scheme rapidly Convergence of ξ to µ in a conditional mean squared sense and with µ probability 1 is proven.

## References

SHOWING 1-10 OF 21 REFERENCES

A new algorithm for the recursive identification of stochastic systems using an automaton with slowly growing memory

- Mathematics, Computer Science1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes
- 1976

It is shown that the BED converges strongly to the correct parameter and for a very large class of systems, convergence of parameter estimates is impossible without acess to an unbounded memory.

On Determining the Irrationality of the Mean of a Random Variable

- Mathematics
- 1973

A complexity approach is used to decide whether or not the mean of a sequence of independent identically distributed random variables lies in a n arbitrary specified countable subset of the real…

Logical basis for information theory and probability theory

- Computer ScienceIEEE Trans. Inf. Theory
- 1968

A new logical basis for information theory as well as probability theory is proposed, based on computing complexity, according to a new approach to computing complexity.

Computational Complexity and Probability Constructions

- Computer ScienceJACM
- 1970

Using any universal Tur ing machine as a basis, it is possible to cons t ruc t an infinite number of increas ingly accurate computable probabil i ty measures which are independen t of any p robab i l i ty assumpt ions.

Some inequalities [lo] A. N. Kohnogordv, humiztiom of the Theory of Probability. New York: Chelsea. 1950. between Shr&on entropy and Kolmogorov, Chaitin and extension comnlexities

- Tech. Reo. Statistics Dem.. Stanford Univ
- 1975

The recursive indentification of stochastic systems using an automaton with slowly growing memory

- presented at IEEE Sym. Inform. Theory, Cornell Univ., Oct. 1977. [12] T. M. Cover, “On the determination of the irrationality of the
- 1977

THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS

- Computer Science
- 1970

The present article is a survey of the fundamental results connected with the concept of complexity as the minimum number of binary signs containing all the information about a given object that are sufficient for its recovery (decoding).

A Theory of Program Size Formally Identical to Information Theory

- Computer ScienceJACM
- 1975

A new definition of program-size complexity is made, which has precisely the formal properties of the entropy concept of information theory.