Corpus ID: 237453215

Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems

  title={Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems},
  author={Fangda Gu and He Yin and Laurent El Ghaoui and Murat Arcak and Peter J. Seiler and Ming Jin},
  • Fangda Gu, He Yin, +3 authors Ming Jin
  • Published 2021
  • Computer Science, Engineering
  • ArXiv
Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability… Expand

Figures from this paper


Robust Reinforcement Learning Control Using Integral Quadratic Constraints for Recurrent Neural Networks
The stability of a control loop including a recurrent neural network (NN) is analyzed by replacing the nonlinear and time-varying components of the NN with IQCs on their gain and an algorithm is demonstrated for training the recurrent NN using reinforcement learning and guaranteeing stability while learning. Expand
Stability-Certified Reinforcement Learning: A Control-Theoretic Perspective
By regulating the partial gradients of policies, strong guarantees of robust stability can be obtained based on a proposed semidefinite programming feasibility problem and the method is able to certify a large set of stabilizing controllers by exploiting problem-specific structures. Expand
Safe Model-based Reinforcement Learning with Stability Guarantees
This paper presents a learning algorithm that explicitly considers safety, defined in terms of stability guarantees, and extends control-theoretic results on Lyapunov stability verification and shows how to use statistical models of the dynamics to obtain high-performance control policies with provable stability certificates. Expand
Stable reinforcement learning with recurrent neural networks
In this paper, we present a technique for ensuring the stability of a large class of adaptively controlled systems. We combine IQC models of both the controlled system and the controller with aExpand
Standard representation and unified stability analysis for dynamic artificial neural network models
An overview is provided of dynamic artificial neural network models (DANNs) for nonlinear dynamical system identification and control problems, and convex stability conditions are proposed that areExpand
A robust stability approach to robot reinforcement learning based on a parameterization of stabilizing controllers
This work proposes to employ reinforcement learning in the context of model-based control, allowing to learn in a framework of stabilizing controllers built by using only little prior model knowledge, so that the action space is suitably structured for safe learning of a feedback controller to compensate for uncertainties due to model mismatch or external disturbances. Expand
Continuous control with deep reinforcement learning
This work presents an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces, and demonstrates that for many of the tasks the algorithm can learn policies end-to-end: directly from raw pixel inputs. Expand
A Lyapunov-based Approach to Safe Reinforcement Learning
This work defines and presents a method for constructing Lyapunov functions, which provide an effective way to guarantee the global safety of a behavior policy during training via a set of local, linear constraints. Expand
Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator
This work bridges the gap showing that (model free) policy gradient methods globally converge to the optimal solution and are efficient (polynomially so in relevant problem dependent quantities) with regards to their sample and computational complexities. Expand
Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming
This paper proposes a semidefinite programming (SDP) framework for safety verification and robustness analysis of neural networks with general activation functions, to abstract various properties of activation functions with the formalism of quadratic constraints. Expand