Corpus ID: 53250107

A Convergence Theory for Deep Learning via Over-Parameterization

@inproceedings{AllenZhu2019ACT,
  title={A Convergence Theory for Deep Learning via Over-Parameterization},
  author={Zeyuan Allen-Zhu and Yuanzhi Li and Zhao Song},
  booktitle={ICML},
  year={2019}
}
  • Zeyuan Allen-Zhu, Yuanzhi Li, Zhao Song
  • Published in ICML 2019
  • Mathematics, Computer Science
  • Deep neural networks (DNNs) have demonstrated dominating performance in many fields; since AlexNet, networks used in practice are going wider and deeper. [...] Key Result In terms of network architectures, our theory at least applies to fully-connected neural networks, convolutional neural networks (CNN), and residual neural networks (ResNet).Expand Abstract

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 313 CITATIONS

    How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?

    VIEW 10 EXCERPTS
    CITES BACKGROUND, METHODS & RESULTS
    HIGHLY INFLUENCED

    A Generalization Theory of Gradient Descent for Learning Over-parameterized Deep ReLU Networks

    • Yuan Cao, Quanquan Gu
    • Mathematics, Computer Science
    • ArXiv
    • 2019
    VIEW 9 EXCERPTS
    CITES BACKGROUND & RESULTS
    HIGHLY INFLUENCED

    An Improved Analysis of Training Over-parameterized Deep Neural Networks

    VIEW 10 EXCERPTS
    CITES BACKGROUND, RESULTS & METHODS
    HIGHLY INFLUENCED

    Generalization Error Bounds of Gradient Descent for Learning Over-Parameterized Deep ReLU Networks

    VIEW 6 EXCERPTS
    CITES BACKGROUND & METHODS
    HIGHLY INFLUENCED

    Training Over-parameterized Deep ResNet Is almost as Easy as Training a Two-layer Network

    VIEW 2 EXCERPTS
    CITES METHODS & BACKGROUND
    HIGHLY INFLUENCED

    Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks

    • Yuan Cao, Quanquan Gu
    • Mathematics, Computer Science
    • NeurIPS
    • 2019
    VIEW 3 EXCERPTS

    Over-parameterization as a Catalyst for Better Generalization of Deep ReLU network

    VIEW 2 EXCERPTS
    CITES BACKGROUND

    FILTER CITATIONS BY YEAR

    2018
    2020

    CITATION STATISTICS

    • 71 Highly Influenced Citations

    • Averaged 105 Citations per year from 2018 through 2020

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 71 REFERENCES