Singular Ridge Regression with Homoscedastic Residuals: Generalization Error with Estimated Parameters

@article{Grigoryeva2016SingularRR,
  title={Singular Ridge Regression with Homoscedastic Residuals: Generalization Error with Estimated Parameters},
  author={Lyudmila Grigoryeva and Juan-Pablo Ortega},
  journal={arXiv: Machine Learning},
  year={2016}
}
This paper characterizes the conditional distribution properties of the finite sample ridge regression estimator and uses that result to evaluate total regression and generalization errors that incorporate the inaccuracies committed at the time of parameter estimation. The paper provides explicit formulas for those errors. Unlike other classical references in this setup, our results take place in a fully singular setup that does not assume the existence of a solution for the non-regularized… Expand
1 Citations

Figures from this paper

Nonlinear Memory Capacity of Parallel Time-Delay Reservoir Computers in the Processing of Multidimensional Signals
TLDR
An approximating reservoir model is presented in frameworks that provides an explicit functional link between the reservoir architecture and its performance in the execution of a specific task, and the inference properties of the ridge regression estimator in the multivariate context are used to assess the impact of finite sample training on the decrease of the reservoir capacity. Expand

References

SHOWING 1-9 OF 9 REFERENCES
Ridge regression: biased estimation for nonorthogonal problems
In multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory, if not incorrect, if the prediction vectors areExpand
Neural Networks and the Bias/Variance Dilemma
TLDR
It is suggested that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Expand
Nonlinear Memory Capacity of Parallel Time-Delay Reservoir Computers in the Processing of Multidimensional Signals
TLDR
An approximating reservoir model is presented in frameworks that provides an explicit functional link between the reservoir architecture and its performance in the execution of a specific task, and the inference properties of the ridge regression estimator in the multivariate context are used to assess the impact of finite sample training on the decrease of the reservoir capacity. Expand
Matrix Variate Distributions
PRELIMINARIES Matrix Algebra Jacobians of Transformations Integration Zonal Polynomials Hypergeometric Functions of Matrix Argument LaGuerre Polynomials Generalized Hermite Polynomials Notion ofExpand
Matrix Analysis and Applied Linear Algebra
TLDR
The author presents Perron-Frobenius theory of nonnegative matrices Index, a theory of matrices that combines linear equations, vector spaces, and matrix algebra with insights into eigenvalues and Eigenvectors. Expand
Time Series Analysis
We provide a concise overview of time series analysis in the time and frequency domains, with lots of references for further reading.