Online Learning and Stochastic Approximations

  title={Online Learning and Stochastic Approximations},
  author={L{\'e}on Bottou},
The convergence of online learning algorithms is analyzed using the tools of the stochastic approximation theory, and proved under very weak conditions. A general framework for online learning algorithms is first presented. This framework encompasses the most common online learning algorithms in use today, as illustrated by several examples. The stochastic approximation theory then provides general results describing the convergence of all these learning algorithms at once. Revised version, May… CONTINUE READING
Highly Influential
This paper has highly influenced 43 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 335 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.


Publications citing this paper.
Showing 1-10 of 231 extracted citations

Deep learning helicopter dynamics models

2015 IEEE International Conference on Robotics and Automation (ICRA) • 2015
View 4 Excerpts
Highly Influenced

Bayesian Nonnegative Matrix Factorization with Stochastic Variational Inference

Handbook of Mixed Membership Models and Their Applications • 2014
View 4 Excerpts
Highly Influenced

336 Citations

Citations per Year
Semantic Scholar estimates that this publication has 336 citations based on the available data.

See our FAQ for additional information.


Publications referenced by this paper.
Showing 1-10 of 33 references

Estimation of dependences based on empirical data

V. N. Vapnik
Springer Series in Statistics. Springer Verlag, Berlin, New York. • 1982
View 4 Excerpts
Highly Influenced

"Foundations of the Theory of Learning Systems, " by Ya. Z. Tsypkin (Book Review)

International Journal of Man-Machine Studies • 1974
View 4 Excerpts
Highly Influenced

Backpropagation Applied to Handwritten Zip Code Recognition

Neural Computation • 1989
View 3 Excerpts
Highly Influenced

Numerical methods for unconstrained optimization and nonlinear equations

Prentice Hall series in computational mathematics • 1983
View 3 Excerpts
Highly Influenced


M. Metivier
Walter de Gruyter, Berlin. • 1983
View 2 Excerpts
Highly Influenced

Adaptation and Learning in automatic systems

Y. Tsypkin
Academic Press, New York. • 1971
View 5 Excerpts
Highly Influenced


M. Minsky, S. Papert
MIT Press, Cambridge, MA. • 1969
View 3 Excerpts
Highly Influenced


D. L. Fisk
Transactions of the American Mathematical Society, 120:359–388. • 1965
View 3 Excerpts
Highly Influenced

Adaptive Switching Circuits

Tifatfj LASSIFIED, -M I, +5 authors ITMItRI MIViiilTV
View 3 Excerpts

Similar Papers

Loading similar papers…