Nonconvergence to Unstable Points in Urn Models and Stochastic Approximations
@article{Pemantle1990NonconvergenceTU, title={Nonconvergence to Unstable Points in Urn Models and Stochastic Approximations}, author={Robin Pemantle}, journal={Annals of Probability}, year={1990}, volume={18}, pages={698-712} }
A particle in Rd moves in discrete time. The size of the nth step is of order 1/n and when the particle is at a position v the expectation of the next step is in the direction F(v) for some fixed vector function F of class C2. It is well known that the only possible points p where v(n) may converge are those satisfying F(p) = 0. This paper proves that convergence to some of these points is in fact impossible as long as the "noise" -the difference between each step and its expectation-is…
239 Citations
Escaping Saddle Points: from Agent-based Models to Stochastic Gradient Descent
- Computer Science, Mathematics
- 2019
This work provides a tight analysis: for any non-attracting fixed point in any stochastic process in this family, it is shown that for a sufficiently small constant > 0, the process will be -far away from the fixed point with high probability in O(n log n) time with high probabilities.
Escaping Saddle Points in Constant Dimensional Spaces: An Agent-based Modeling Perspective
- Computer Science, MathematicsEC
- 2020
For any non-attracting hyperbolic fixed point and a sufficiently small constant ε >0, the process will be ε-far away from the fixed point in O(nlog n) time with high probability, and it takes time Ω(n log n) to escape such a fixed point with constant probability, showing that the result is optimal up to a multiplicative constant.
Stability and instability of limit points for stochastic approximation algorithms
- MathematicsIEEE Trans. Autom. Control.
- 2000
It is shown that the limit points of a stochastic approximation (SA) algorithm consist of a connected set and sufficient conditions are given for stability of x/spl dot/=f(x) at x~ if {x/sub n/} tends to x~ for any initial values.
On the fast convergence of random perturbations of the gradient flow
- Computer Science, MathematicsAsymptotic Analysis
- 2017
It is proved that under mild conditions, when the potential function is a Morse function with additional strong saddle condition, the perturbed gradient flow converges to the neighborhood of local minimizers in O ( ln ( ε − 1 ) ) time on the average, where ε is the scale of the random perturbation.
Stochastic optimization with momentum: Convergence, fluctuations, and traps avoidance
- Mathematics
- 2020
In this paper, a general stochastic optimization procedure is studied, unifying several variants of the stochastic gradient descent such as, among others, the stochastic heavy ball method, the…
Dynamics of stochastic approximation with Markov iterate-dependent noise with the stability of the iterates not ensured
- Mathematics, Computer Science
- 2016
This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand by extending the lock-in probability to such recursions, and uses these results to prove almost sure convergence of the Iterates to the specified attractor when theIterates satisfy an asymptotic tightness condition.
A Dynamical System Approach to Stochastic Approximations
- Mathematics
- 1996
It is known that some problems of almost sure convergence for stochastic approximation processes can be analyzed via an ordinary differential equation (ODE) obtained by suitable averaging. The goal…
Dynamics of Morse-Smale urn processes
- MathematicsErgodic Theory and Dynamical Systems
- 1995
Abstract We consider stochastic processes {xn}n≥0 of the form where F: ℝm → ℝm is C2, {λi}i≥1 is a sequence of positive numbers decreasing to 0 and {Ui}i≥1 is a sequence of uniformly bounded…
Stochastic Subgradient Descent Escapes Active Strict Saddles
- Mathematics, Computer Science
- 2021
In non-smooth stochastic optimization, we establish the non-convergence of the stochastic subgradient descent (SGD) to the critical points recently called active strict saddles by Davis and…
Attainability of boundary points under reinforcement learning
- EconomicsGames Econ. Behav.
- 2005
References
SHOWING 1-4 OF 4 REFERENCES
Stochastic Approximation and Recursive Estimation
- Art
- 1976
An extrusion head for an extrudable material, such as a food product dough, which is to be formed into a special shape, such as a pretzel configuration. The head includes a die and associated parts…