Nonconvergence to Unstable Points in Urn Models and Stochastic Approximations

  title={Nonconvergence to Unstable Points in Urn Models and Stochastic Approximations},
  author={Robin Pemantle},
  journal={Annals of Probability},
  • R. Pemantle
  • Published 1 April 1990
  • Mathematics
  • Annals of Probability
A particle in Rd moves in discrete time. The size of the nth step is of order 1/n and when the particle is at a position v the expectation of the next step is in the direction F(v) for some fixed vector function F of class C2. It is well known that the only possible points p where v(n) may converge are those satisfying F(p) = 0. This paper proves that convergence to some of these points is in fact impossible as long as the "noise" -the difference between each step and its expectation-is… 
Escaping Saddle Points: from Agent-based Models to Stochastic Gradient Descent
This work provides a tight analysis: for any non-attracting fixed point in any stochastic process in this family, it is shown that for a sufficiently small constant > 0, the process will be -far away from the fixed point with high probability in O(n log n) time with high probabilities.
Escaping Saddle Points in Constant Dimensional Spaces: An Agent-based Modeling Perspective
For any non-attracting hyperbolic fixed point and a sufficiently small constant ε >0, the process will be ε-far away from the fixed point in O(nlog n) time with high probability, and it takes time Ω(n log n) to escape such a fixed point with constant probability, showing that the result is optimal up to a multiplicative constant.
Stability and instability of limit points for stochastic approximation algorithms
It is shown that the limit points of a stochastic approximation (SA) algorithm consist of a connected set and sufficient conditions are given for stability of x/spl dot/=f(x) at x~ if {x/sub n/} tends to x~ for any initial values.
On the fast convergence of random perturbations of the gradient flow
It is proved that under mild conditions, when the potential function is a Morse function with additional strong saddle condition, the perturbed gradient flow converges to the neighborhood of local minimizers in O ( ln ( ε − 1 ) ) time on the average, where ε is the scale of the random perturbation.
Stochastic optimization with momentum: Convergence, fluctuations, and traps avoidance
In this paper, a general stochastic optimization procedure is studied, unifying several variants of the stochastic gradient descent such as, among others, the stochastic heavy ball method, the
Dynamics of stochastic approximation with Markov iterate-dependent noise with the stability of the iterates not ensured
This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand by extending the lock-in probability to such recursions, and uses these results to prove almost sure convergence of the Iterates to the specified attractor when theIterates satisfy an asymptotic tightness condition.
A Dynamical System Approach to Stochastic Approximations
It is known that some problems of almost sure convergence for stochastic approximation processes can be analyzed via an ordinary differential equation (ODE) obtained by suitable averaging. The goal
Dynamics of Morse-Smale urn processes
Abstract We consider stochastic processes {xn}n≥0 of the form where F: ℝm → ℝm is C2, {λi}i≥1 is a sequence of positive numbers decreasing to 0 and {Ui}i≥1 is a sequence of uniformly bounded
Stochastic Subgradient Descent Escapes Active Strict Saddles
In non-smooth stochastic optimization, we establish the non-convergence of the stochastic subgradient descent (SGD) to the critical points recently called active strict saddles by Davis and
Attainability of boundary points under reinforcement learning


Differential Equations, Dynamical Systems, and Linear Algebra
Stochastic Approximation and Recursive Estimation
An extrusion head for an extrudable material, such as a food product dough, which is to be formed into a special shape, such as a pretzel configuration. The head includes a die and associated parts