Corpus ID: 232076092

Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networks

@article{Nakada2021AsymptoticRO,
  title={Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networks},
  author={Ryumei Nakada and M. Imaizumi},
  journal={ArXiv},
  year={2021},
  volume={abs/2103.00500}
}
We investigate the asymptotic risk of a general class of overparameterized likelihood models, including deep models. The recent empirical success of large-scale models has motivated several theoretical studies to investigate a scenario wherein both the number of samples, n, and parameters, p, diverge to infinity and derive an asymptotic risk at the limit. However, these theorems are only valid for linear-in-feature models, such as generalized linear regression, kernel regression, and shallow… Expand

Figures from this paper

References

SHOWING 1-10 OF 51 REFERENCES
Generalization of Two-layer Neural Networks: An Asymptotic Viewpoint
  • 25
Benign overfitting in ridge regression
  • 10
  • PDF
High-dimensional dynamics of generalization error in neural networks
  • 194
  • PDF
Regularization in High-Dimensional Regression and Classification via Random Matrix Theory
  • 5
  • PDF
A Model of Double Descent for High-dimensional Binary Linear Classification
  • 50
  • PDF
High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification
  • 95
  • Highly Influential
  • PDF
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime
  • 38
  • PDF
Two models of double descent for weak features
  • 141
  • PDF
Triple descent and the two kinds of overfitting: Where & why do they appear?
  • 12
  • PDF
...
1
2
3
4
5
...