The Minimum Description Length Principle in Coding and Modeling

@article{Barron1998TheMD,
  title={The Minimum Description Length Principle in Coding and Modeling},
  author={A. Barron and J. Rissanen and Bin Yu},
  journal={IEEE Trans. Inf. Theory},
  year={1998},
  volume={44},
  pages={2743-2760}
}
We review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling. Stochastic complexity is formulated as the solution to optimum universal coding problems extending Shannon's basic source coding theorem. The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms. We assess the performance of the minimum description length… Expand
M Minimum Description Length Principle
Model selection by normalized maximum likelihood
Normalized Maximum Likelihood Models for Discrete Regression
Minimum description length induction, Bayesianism, and Kolmogorov complexity
A generalized minmax bound for universal coding
  • J. Rissanen
  • Mathematics
  • 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060)
  • 2000
Researches on Properties of Generalized Normalization Coefficient in NML Distribution
  • Fucai Zhou, Ping Sun, Jinying Wang
  • Mathematics, Computer Science
  • First International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06)
  • 2006
Minimum Redundancy Coding for Uncertain Sources
Bayesian properties of normalized maximum likelihood and its fast computation
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 76 REFERENCES
Minimum complexity density estimation
Universal coding, information, prediction, and estimation
  • J. Rissanen
  • Mathematics, Computer Science
  • IEEE Trans. Inf. Theory
  • 1984
Asymptotically optimal function estimation by minimum complexity criteria
Universal modeling and coding
Universal noiseless coding
  • L. Davisson
  • Mathematics, Computer Science
  • IEEE Trans. Inf. Theory
  • 1973
A strong version of the redundancy-capacity theorem of universal coding
Fisher information and stochastic complexity
  • J. Rissanen
  • Mathematics, Computer Science
  • IEEE Trans. Inf. Theory
  • 1996
Data compression and histograms
Minimax noiseless universal coding for Markov sources
  • L. Davisson
  • Mathematics, Computer Science
  • IEEE Trans. Inf. Theory
  • 1983
The context-tree weighting method: basic properties
...
1
2
3
4
5
...