THE MDL PRINCIPLE , PENALIZED LIKELIHOODS , AND STATISTICAL RISK

@inproceedings{Barron2008THEMP,
  title={THE MDL PRINCIPLE , PENALIZED LIKELIHOODS , AND STATISTICAL RISK},
  author={Andrew R. Barron and Cong Huang and LI JONATHANQ. and Xi Bo Luo and A. Bstract},
  year={2008}
}
We determine, for both countable and uncountable collections of functions, informationtheoretic conditions on a penalty pen(f) such that the optimizer f̂ of the penalized log likelihood criterion log 1/likelihood(f) + pen(f) has statistical risk not more than the index of resolvability corresponding to the accuracy of the optimizer of the expected value of the criterion. If F is the linear span of a dictionary of functions, traditional description-length penalties are based on the number of non… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 10 CITATIONS

A NEW METHOD FOR COVARIATE SELECTION IN COX MODEL

VIEW 4 EXCERPTS
CITES METHODS
HIGHLY INFLUENCED

Information theoretic validity of penalized likelihood

  • 2014 IEEE International Symposium on Information Theory
  • 2014
VIEW 5 EXCERPTS
CITES BACKGROUND

Finite-Sample Risk Bounds for Maximum Likelihood Estimation With Arbitrary Penalties

  • IEEE Transactions on Information Theory
  • 2017
VIEW 3 EXCERPTS
CITES BACKGROUND & METHODS

Minimax lower bounds for ridge combinations including neural nets

  • 2017 IEEE International Symposium on Information Theory (ISIT)
  • 2017
VIEW 2 EXCERPTS
CITES METHODS & BACKGROUND

Noisy inductive matrix completion under sparse factor models

  • 2017 IEEE International Symposium on Information Theory (ISIT)
  • 2016
VIEW 1 EXCERPT
CITES BACKGROUND

MDL, penalized likelihood, and statistical risk

  • 2008 IEEE Information Theory Workshop
  • 2008
VIEW 2 EXCERPTS
CITES METHODS & BACKGROUND

References

Publications referenced by this paper.
SHOWING 1-10 OF 82 REFERENCES

Risk of penalized least squares, greedy selection and `1-penalization from flexible function libraries

C. Huang, G.H.L. Cheang, A. R. Barron
  • Submitted to Annals of Statistics. Jones, L. K. (1992). A simple lemma on greedy approximation in Hilbert spaces and convergence rates for projection pursuit regression and neural network training. Annals of Statistics, Vol. 20, p. 608-613.
  • 2008
VIEW 8 EXCERPTS
HIGHLY INFLUENTIAL

The Minimum Description Length Principle

VIEW 6 EXCERPTS
HIGHLY INFLUENTIAL

Minimum complexity density estimation

  • IEEE Trans. Information Theory
  • 1991
VIEW 11 EXCERPTS

The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions

A. R. Barron
  • University of Illinois Department of Statistics Technical Report #7. Available at www.stat.yale.edu/∼arb4/publications.htm
  • 1988
VIEW 10 EXCERPTS
HIGHLY INFLUENTIAL