A risk comparison of ordinary least squares vs ridge regression

  title={A risk comparison of ordinary least squares vs ridge regression},
  author={Paramveer S. Dhillon and Dean P. Foster and Sham M. Kakade and Lyle H. Ungar},
  journal={Journal of Machine Learning Research},
We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a principal component analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that the risk of this ordinary least squares method (PCA-OLS) is within a constant factor (namely 4) of the risk of ridge regression (RR). 
Highly Cited
This paper has 30 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.


Publications citing this paper.
Showing 1-10 of 20 extracted citations


Publications referenced by this paper.
Showing 1-2 of 2 references

The risk inflation criterion for multiple regression

  • D. P. Foster, E. I. George
  • The Annals of Statistics,
  • 1994
1 Excerpt

Solution of incorrectly formulated problems and the regularization method

  • A. N. Tikhonov
  • Soviet Math Dokl
  • 1963
2 Excerpts

Similar Papers

Loading similar papers…