On the momentum term in gradient descent learning algorithms

@article{Qian1999OnTM,
  title={On the momentum term in gradient descent learning algorithms},
  author={Ning Qian},
  journal={Neural networks : the official journal of the International Neural Network Society},
  year={1999},
  volume={12 1},
  pages={
          145-151
        }
}
A momentum term is usually included in the simulations of connectionist learning algorithms. Although it is well known that such a term greatly improves the speed of learning, there have been few rigorous studies of its mechanisms. In this paper, I show that in the limit of continuous time, the momentum parameter is analogous to the mass of Newtonian particles that move through a viscous medium in a conservative force field. The behavior of the system near a local minimum is equivalent to a set… CONTINUE READING
BETA

From This Paper

Figures, tables, and topics from this paper.

Citations

Publications citing this paper.
SHOWING 1-10 OF 284 CITATIONS, ESTIMATED 44% COVERAGE

646 Citations

0100200'01'04'08'12'16
Citations per Year
Semantic Scholar estimates that this publication has 646 citations based on the available data.

See our FAQ for additional information.

Similar Papers

Loading similar papers…