Corpus ID: 605683

Effect of Batch Learning in Multilayer Neural Networks

  title={Effect of Batch Learning in Multilayer Neural Networks},
  author={K. Fukumizu},
  • K. Fukumizu
  • Published in ICONIP 1998
  • Computer Science
  • This paper discusses batch gradient descent learning in mul-tilayer networks with a large number of statistical training data. We emphasize on the diierence between regular cases, where the prepared model has the same size as the true function , and overrealizable cases, where the model has surplus hidden units to realize the true function. First, experimental study on multilayer perceptrons and linear neural networks (LNN) shows that batch learning induces strong overtrain-ing on both models… CONTINUE READING

    Topics from this paper.

    Dynamics of Batch Learning in Multilayer Neural Networks
    • 2
    • PDF
    Layer Dynamics of Linearised Neural Nets
    On the Information Bottleneck Theory of Deep Learning
    • 162
    • PDF
    Minnorm training: an algorithm for training over-parameterized deep neural networks
    • 10
    • PDF
    Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
    • 918
    • PDF
    On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
    • 148
    • Highly Influenced
    • PDF


    Publications referenced by this paper.
    Learning in linear neural networks: a survey
    • 245
    • PDF
    A Regularity Condition of the Information Matrix of a Multilayer Perceptron Network
    • 75
    Special Statistical Properties of Neural Network Learning
    • 7
    • PDF
    Universal approximation bounds for superpositions of a sigmoidal function
    • 2,249
    • Highly Influential
    • PDF
    Global analysis of Oja's flow for neural networks
    • 102
    Simplified neuron model as a principal component analyzer
    • E. Oja
    • Mathematics, Medicine
    • 1982
    • 2,002
    • PDF
    Statistical the- ory of overtraining { is cross-validation asymptotically e ective?," Advances in Neural Information Processing Systems 8, pp.176{182
    • 1996