# Prediction Errors for Penalized Regressions based on Generalized Approximate Message Passing

@article{Sakata2022PredictionEF, title={Prediction Errors for Penalized Regressions based on Generalized Approximate Message Passing}, author={Ayaka Sakata}, journal={ArXiv}, year={2022}, volume={abs/2206.12832} }

We discuss the prediction accuracy of assumed statistical models in terms of prediction errors for the generalized linear model and penalized maximum likelihood methods. We derive the forms of estimators for the prediction errors: Cp criterion, information criteria, and leave-one-out cross validation (LOOCV) error, using the generalized approximate message passing (GAMP) algorithm and replica method. These estimators coincide with each other when the number of model parameters is sufficiently…

## Figures and Tables from this paper

## References

SHOWING 1-10 OF 18 REFERENCES

### Statistical Learning with Sparsity: The Lasso and Generalizations

- Computer Science
- 2015

Statistical Learning with Sparsity: The Lasso and Generalizations presents methods that exploit sparsity to help recover the underlying signal in a set of data and extract useful and reproducible patterns from big datasets.

### Asymptotic Errors for High-Dimensional Convex Penalized Linear Regression beyond Gaussian Matrices.

- Mathematics, Computer ScienceCOLT 2020
- 2020

A rigorous derivation of an explicit formula for the asymptotic mean squared error obtained by penalized convex regression estimators such as the LASSO or the elastic net is provided, for a class of very generic random matrices corresponding to rotationally invariant data matrices with arbitrary spectrum.

### Generalized approximate message passing for estimation with random linear mixing

- Computer Science2011 IEEE International Symposium on Information Theory Proceedings
- 2011

G-AMP incorporates general measurement channels and shows that the asymptotic behavior of the G-AMP algorithm under large i.i.d. measurement channels is similar to the AWGN output channel case, and Gaussian transform matrices is described by a simple set of state evolution (SE) equations.

### On the interplay between noise and curvature and its effect on optimization and generalization

- Computer ScienceAISTATS
- 2020

This work clarifies the distinction between the Fisher matrix, the Hessian, and the covariance matrix of the gradients and explains how both curvature and noise are relevant to properly estimate the generalization gap.

### Information Theory and an Extension of the Maximum Likelihood Principle

- Mathematics
- 1973

The classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion to provide answers to many practical problems of statistical model fitting.

### Probabilistic reasoning in intelligent systems - networks of plausible inference

- Computer ScienceMorgan Kaufmann series in representation and reasoning
- 1989

The author provides a coherent explication of probability as a language for reasoning with partial belief and offers a unifying perspective on other AI approaches to uncertainty, such as the Dempster-Shafer formalism, truth maintenance systems, and nonmonotonic logic.

### Algebraic Geometry and Statistical Learning Theory

- Computer Science
- 2009

The theory achieved here underpins accurate estimation techniques in the presence of singularities and lays the foundations for the use of algebraic geometry in statistical learning theory.

### Statistical physics of spin glasses and information processing : an introduction

- Physics
- 2001

1. Mean-field theory of phase transitions 2. Mean-field theory of spin glasses 3. Replica symmetry breaking 4. Gauge theory of spin glasses 5. Error-correcting codes 6. Image restoration 7.…

### The Elements of Statistical Learning (Springer Science & Business Media

- 2009