Corpus ID: 2133959

On the Computational Efficiency of Training Neural Networks

@inproceedings{Livni2014OnTC,
  title={On the Computational Efficiency of Training Neural Networks},
  author={Roi Livni and S. Shalev-Shwartz and O. Shamir},
  booktitle={NIPS},
  year={2014}
}
  • Roi Livni, S. Shalev-Shwartz, O. Shamir
  • Published in NIPS 2014
  • Computer Science, Mathematics
  • It is well-known that neural networks are computationally hard to train. [...] Key Result We provide both positive and negative results, some of them yield new provably efficient and practical algorithms for training certain types of neural networks.Expand Abstract

    Paper Mentions

    Understanding deep learning requires rethinking generalization
    • 2,045
    • Open Access
    Convergence Analysis of Two-layer Neural Networks with ReLU Activation
    • 292
    • Open Access
    Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
    • 247
    • Open Access
    A Convergence Theory for Deep Learning via Over-Parameterization
    • 378
    • Open Access
    AdaNet: Adaptive Structural Learning of Artificial Neural Networks
    • 142
    • Open Access
    Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data
    • 251
    • Open Access
    Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
    • 208
    • Open Access
    On the Quality of the Initial Basin in Overspecified Neural Networks
    • 98
    • Open Access
    Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs
    • 210
    • Highly Influenced
    • Open Access
    Breaking the Curse of Dimensionality with Convex Neural Networks
    • 256
    • Highly Influenced
    • Open Access

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 28 REFERENCES
    ImageNet classification with deep convolutional neural networks
    • 51,808
    • Open Access
    ImageNet Classification with Deep Convolutional Neural Networks
    • 51,808
    • Open Access
    Representation Learning: A Review and New Perspectives
    • 6,176
    • Open Access
    On the importance of initialization and momentum in deep learning
    • 2,490
    • Open Access
    Training a 3-node neural network is NP-complete
    • 716
    • Open Access
    Neural Network Learning - Theoretical Foundations
    • 1,328
    • Open Access
    Visualizing and Understanding Convolutional Networks
    • 8,254
    • Open Access
    Hardness results for neural network approximation problems
    • 42
    • Open Access
    Learning Polynomials with Neural Networks
    • 110
    • Open Access
    Building high-level features using large scale unsupervised learning
    • 1,870
    • Open Access