Ill-Conditioning in Neural Network Training Problems

  title={Ill-Conditioning in Neural Network Training Problems},
  author={Sirpa Saarinen and Randall Bramley and George Cybenko},
  journal={SIAM J. Scientific Computing},
The training problem for feedforward neural networks is nonlinear parameter estimation that can be solved by a variety of optimization techniques. Much of the literature on neural networks has focused on variants of gradient descent. The training of neural networks using such techniques is known to be a slow process with more sophisticated techniques not always performing signiicantly better. In this paper, we show that feedforward neural networks can have ill-conditioned Hessians and that this… CONTINUE READING
Highly Cited
This paper has 184 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.


Publications citing this paper.
Showing 1-10 of 69 citations

184 Citations

Citations per Year
Semantic Scholar estimates that this publication has 184 citations based on the available data.

See our FAQ for additional information.


Publications referenced by this paper.
Showing 1-10 of 17 references

Neural networks, backpropagation, and automatic diierentiation, in Automatic Diierentiation of Algorithms: Theory, Implementation, and Application

  • S Saarinen, R Bramley, G Cybenko
  • Neural networks, backpropagation, and automatic…
  • 1992

Matrix Computations

  • G Golub, C Van, Loan
  • Matrix Computations
  • 1989

Multivariate interpolation in odd dimensional euclidean spaces using multiquadratics

  • M Buhmann
  • Dept. of Appl. Math. and Theor. Physics
  • 1988

Radial basis functions for multivariable interpolation: a review, in IMA Conference on Algorithms for the Approximation of Functions and Data

  • M Powell
  • Radial basis functions for multivariable…
  • 1987