Stochastic Complexity in Learning

  title={Stochastic Complexity in Learning},
  author={Jorma Rissanen},
  journal={J. Comput. Syst. Sci.},
This is an expository paper on the latest results in the theory of stochastic complexity and the associated MDL principle with special interest in modeling problems arising in machine learning. As an illustration we discuss the problem of designing MDL decision trees, which are meant to improve the earlier designs in two ways: First, by use of the sharper formula for the stochastic complexity at the nodes the earlier found tendency of getting too small trees appears to be overcome. Second, a… CONTINUE READING

From This Paper

Topics from this paper.
2 Citations
20 References
Similar Papers


Publications citing this paper.
Showing 1-2 of 2 extracted citations


Publications referenced by this paper.
Showing 1-10 of 20 references

Information theory and neural nets, in ``Mathematical Perspectives of Neural Networks'' (P

  • J. Rissanen
  • Smolensky, M. Mozer, and D. Rumelhart, Eds…
  • 1992
Highly Influential
11 Excerpts

Three approaches to the quantitative definition of information, Probl

  • A. N. Kolmogorov
  • Inform. Transmission
  • 1965
Highly Influential
5 Excerpts

The performance of universal coding

  • R. E. Krichevsky, V. K. Trofimov
  • IEEE Trans. Inform. Theory IT-27,
  • 1983
Highly Influential
2 Excerpts

MDL learning, in ``The Workshop Learning and Geometry: The Proceedings, Learning and Geometry: Computational Approaches'' (D

  • J. Rissanen, B. Yu
  • W. Kueker and C. H. Smith, Eds.), pp. 3 19…
  • 1996
1 Excerpt

``Some Topics in Descriptive Complexity,'

  • R. Nohre
  • Ph.D. thesis, Linkoping University,
  • 1994
3 Excerpts

Similar Papers

Loading similar papers…