A DAPTIVITY OF AVERAGED STOCHASTIC GRADIENT DESCENT Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression

@inproceedings{BACH2013ADO,
  title={A DAPTIVITY OF AVERAGED STOCHASTIC GRADIENT DESCENT Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression},
  author={FRANCIS. BACH},
  year={2013}
}
In this paper, we consider supervised learning problems such as logistic regression and study the stochastic gradient method with averaging, in the usual stochastic approximation setting where observations are used only once. We show that after N iterations, with a constant step-size proportional to 1/R √ N where N is the number of observations and R is the maximum norm of the observations, the convergence rate is always of order O(1/ √ N), and improves to O(R/μN) where μ is the lowest… CONTINUE READING