Learning Stabilizing Policies in Stochastic Control Systems

@article{Zikelic2022LearningSP,
  title={Learning Stabilizing Policies in Stochastic Control Systems},
  author={Dorde Zikelic and Mathias Lechner and Krishnendu Chatterjee and Thomas A. Henzinger},
  journal={ArXiv},
  year={2022},
  volume={abs/2205.11991}
}
In this work, we address the problem of learning provably stable neural network policies for stochastic control systems. While recent work has demonstrated the feasibility of certifying given policies using martingale theory, the problem of how to learn such policies is little explored. Here, we study the effectiveness of jointly learning a policy together with a martingale certificate that proves its stability using a single learning algorithm. We observe that the joint optimization problem… 
[PDF] Semantic Reader
1 Citation
Methods Citations
1

Tables from this paper

One Citation

Learning Control Policies for Region Stabilization in Stochastic Systems

This work considers the problem of learning control policies in stochastic systems which guarantee that the system stabilizes within some specified stabilization region with probability 1 and presents a learning procedure that learns a control policy together with an sRSM that formally certifies probability- 1 stability, both learned as neural networks.

22 References

Safe Model-based Reinforcement Learning with Stability Guarantees

This paper presents a learning algorithm that explicitly considers safety, defined in terms of stability guarantees, and extends control-theoretic results on Lyapunov stability verification and shows how to use statistical models of the dynamics to obtain high-performance control policies with provable stability certificates.

The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamic Systems

A method to learn accurate safety certificates for nonlinear, closed-loop dynamical systems by constructing a neural network Lyapunov function and a training algorithm that adapts it to the shape of the largest safe region in the state space.

Neural Lyapunov Control

The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness guarantee, and can obtain much larger regions of attraction than existing methods such as LQR and SOS/SDP.

Stability Verification in Stochastic Control Systems via Neural Network Supermartingales

This work uses ranking supermartingales (RSMs) to certify a.s. asymptotic stability of the system and provides the first method to obtain bounds on the stabilization time, which stochastic Lyapunov functions do not.

Stochastic optimal control via Bellman's principle

Proximal Policy Optimization Algorithms

We propose a new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective…

A comprehensive survey on safe reinforcement learning

This work categorize and analyze two approaches of Safe Reinforcement Learning, based on the modification of the optimality criterion, the classic discounted finite/infinite horizon, with a safety factor and the incorporation of external knowledge or the guidance of a risk metric.

SReachTools: a MATLAB stochastic reachability toolbox

SReachTools implements several new algorithms based on convex optimization, computational geometry, and Fourier transforms, to efficiently compute over- and under-approximations of the stochastic reach set.

A partial history of the early development of continuous-time nonlinear stochastic systems theory

Stochastic stability analysis of discrete-time system using Lyapunov measure

    U. Vaidya
    Mathematics
    2015 American Control Conference (ACC)
  • 2015
A linear transfer operator-based Lyapunov measure is introduced as a new tool for stability verification of stochastic systems, using the duality property between the linear transfer Perron-Frobenius and Koopman operators.