• Corpus ID: 208075795

Safe Interactive Model-Based Learning

  title={Safe Interactive Model-Based Learning},
  author={Marco Gallieri and Seyed Sina Mirrazavi Salehian and Nihat Engin Toklu and Alessio Quaglino and Jonathan Masci and Jan Koutn'ik and Faustino J. Gomez},
Control applications present hard operational constraints. A violation of this can result in unsafe behavior. This paper introduces Safe Interactive Model Based Learning (SiMBL), a framework to refine an existing controller and a system model while operating on the real environment. SiMBL is composed of the following trainable components: a Lyapunov function, which determines a safe set; a safe control policy; and a Bayesian RNN forward model. A min-max control framework, based on alternate… 

Figures from this paper

Bayesian Safe Learning and Control with Sum-of-Squares Analysis and Polynomial Kernels

An iterative method to safely learn the unmodeled dynamics of a control-affine nonlinear system using Bayesian Gaussian process (GP) models with polynomial kernel functions with sum-of-squares programming is proposed.

Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples

The proposed approach can improve upon the initial demonstrator also in practice and achieve better stability than popular reinforcement learning baselines, and is demonstrated on non-linear continuous control tasks with soft constraints.

Neural Lyapunov Model Predictive Control

Results demonstrate that, when a neural network trained on short sequences is used for predictions, a one-step horizon Neural Lyapunov MPC can successfully reproduce the expert behaviour and significantly outperform longer horizon MPCs.

Learning for Safety-Critical Control with Control Barrier Functions

A machine learning framework utilizing Control Barrier Functions (CBFs) to reduce model uncertainty as it impact the safe behavior of a system, ultimately achieving safe behavior.

Furthermore , the exploitation of multi-layer neural networks as function approximations

  • Computer Science, Mathematics
  • 2020
This paper will combine Lyapunov’s method in control theory and stochastic analysis to analyze the mean square stability of MDP in a model-free manner and proposes a REINFORCE-like algorithm to learn the controller and the LyAPunov function simultaneously.

Neural Stochastic Control

Two types of controllers are proposed, viz., the exponential stabilizer (ES) based on the stochastic Lyapunov theory and the asymptotic stabilizers (AS)based on the Stochastic asymPTotic stability theory, which are complementary in applications.

Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics

New methods for learning neural control policies and neural Lyapunov critic functions in the modelfree reinforcement learning (RL) setting are developed to enhance stability of neural controllers for various nonlinear systems including automobile and quadrotor control.



Learning-Based Model Predictive Control for Safe Exploration

This paper presents a learning-based model predictive control scheme that can provide provable high-probability safety guarantees and exploits regularity assumptions on the dynamics in terms of a Gaussian process prior to construct provably accurate confidence intervals on predicted trajectories.

Reachability-based safe learning with Gaussian processes

This work proposes a novel method that uses a principled approach to learn the system's unknown dynamics based on a Gaussian process model and iteratively approximates the maximal safe set and further incorporates safety into the reinforcement learning performance metric, allowing a better integration of safety and learning.

Gaussian Processes in Reinforcement Learning: Stability Analysis and Efficient Value Propagation

Two current limitations ofmodel-based RL that are indispensable prerequisites for widespread deployment of model- based RL in real world tasks are addressed and an approximation based on numerical quadrature that can handle complex state distributions is proposed.

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

This work proposes a controller architecture that combines a model-free RL-based controller with model-based controllers utilizing control barrier functions (CBFs) and on-line learning of the unknown system dynamics, in order to ensure safety during learning.

Gaussian Processes for Data-Efficient Learning in Robotics and Control

This paper learns a probabilistic, non-parametric Gaussian process transition model of the system and applies it to autonomous learning in real robot and control tasks, achieving an unprecedented speed of learning.

Towards Safe Reinforcement Learning Using NMPC and Policy Gradients: Part II - Deterministic Case

We present a methodology to deploy the stochastic policy gradient method, using actor-critic techniques, when the optimal policy is approximated using a parametric optimization problem, allowing one

The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamic Systems

A method to learn accurate safety certificates for nonlinear, closed-loop dynamical systems by constructing a neural network Lyapunov function and a training algorithm that adapts it to the shape of the largest safe region in the state space.

Safe Model-based Reinforcement Learning with Stability Guarantees

This paper presents a learning algorithm that explicitly considers safety, defined in terms of stability guarantees, and extends control-theoretic results on Lyapunov stability verification and shows how to use statistical models of the dynamics to obtain high-performance control policies with provable stability certificates.

A Lyapunov-based Approach to Safe Reinforcement Learning

This work defines and presents a method for constructing Lyapunov functions, which provide an effective way to guarantee the global safety of a behavior policy during training via a set of local, linear constraints.

Model Predictive Control: Classical, Robust and Stochastic

Graduate students pursuing courses in model predictive control or more generally in advanced or process control and senior undergraduates in need of a specialized treatment will find Model Predictive Control an invaluable guide to the state of the art in this important subject.