# Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: A stochastic control approach

@article{Breiten2021StochasticGD, title={Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: A stochastic control approach}, author={Tobias Breiten and Carsten Hartmann and Lara Neureither and Upanshu Sharma}, journal={Journal of Mathematical Physics}, year={2021} }

We study the convergence to equilibrium of an underdamped Langevin equation that is controlled by a linear feedback force. Specifically, we are interested in sampling the possibly multimodal invariant probability distribution of a Langevin system at small noise (or low temperature), for which the dynamics can easily get trapped inside metastable subsets of the phase space. We follow [Chen et al., J. Math. Phys. 56, 113302, 2015] and consider a Langevin equation that is simulated at a high…

## 3 Citations

### Choice of damping coefficient in Langevin dynamics

- MathematicsThe European Physical Journal B
- 2021

This article considers the application of Langevin dynamics to sampling and investigates how to choose the damping parameter in Langevin dynamics for the purpose of maximizing thoroughness of…

### Improving the Convergence Rates for the Kinetic Fokker-Planck Equation by Optimal Control

- Mathematics
- 2022

The long time behavior and detailed convergence analysis of Langevin equations has received increased attention over the last years. Diﬃculties arise from a lack of coercivity, usually termed…

### Poisson Equations with locally-Lipschitz coefficients and Uniform in Time Averaging for Stochastic Differential Equations via Strong Exponential Stability

- Mathematics
- 2022

We study Poisson equations and averaging for Stochastic Differential Equations (SDEs). Poisson equations are essential tools in both probability theory and partial differential equations (PDEs).…

## References

SHOWING 1-10 OF 55 REFERENCES

### Fast cooling for a system of stochastic oscillators

- MathematicsArXiv
- 2014

Among the feedback controls achieving the desired asymptotically driving the system to a desired steady state corresponding to reduced thermal noise, the most efficient one from an energy point of view is characterized by time-reversibility.

### Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions

- Mathematics, Computer ScienceJournal of statistical physics
- 2017

It is shown that appropriate choices of the perturbations can lead to samplers that have improved properties, at least in terms of reducing the asymptotic variance, and a detailed analysis of the new Langevin sampler for Gaussian target distributions is presented.

### CoolMomentum: a method for stochastic optimization by Langevin dynamics with simulated annealing

- Computer Science, PhysicsScientific reports
- 2021

It is shown that a gradual decrease of the momentum coefficient from the initial value close to unity until zero is equivalent to application of Simulated Annealing or slow cooling, in physical terms.

### Partitioned integrators for thermodynamic parameterization of neural networks

- Computer ScienceFoundations of Data Science
- 2019

Evidence is presented that thermodynamic parameterization methods can be faster, more accurate, and more robust than standard algorithms incorporated into machine learning frameworks, in particular for data sets with complicated loss landscapes.

### Mean-field Langevin dynamics and energy landscape of neural networks

- MathematicsAnnales de l'Institut Henri Poincaré, Probabilités et Statistiques
- 2021

We present a probabilistic analysis of the long-time behaviour of the nonlocal, diffusive equations with a gradient flow structure in 2-Wasserstein metric, namely, the Mean-Field Langevin Dynamics…