Accelerated Continuous-Time Approximate Dynamic Programming via Data-Assisted Hybrid Control

@article{Ochoa2022AcceleratedCA,
  title={Accelerated Continuous-Time Approximate Dynamic Programming via Data-Assisted Hybrid Control},
  author={Daniel E. Ochoa and Jorge I. Poveda},
  journal={ArXiv},
  year={2022},
  volume={abs/2204.12707}
}
We introduce a new closed-loop architecture for the online solution of approximate optimal control problems in the context of continuous-time systems. Specifically, we introduce the first algorithm that incorporates dynamic momentum in actor-critic structures to control continuous-time dynamic plants with an affine structure in the input. By incorporating dynamic momentum in our algorithm, we are able to accelerate the convergence properties of the closed-loop system, achieving superior transient… 
[PDF] Semantic Reader

Figures from this paper

References

SHOWING 1-10 OF 24 REFERENCES

Robust Hybrid Zero-Order Optimization Algorithms with Acceleration via Averaging in Continuous Time

A family of derivative-free dynamics that can be universally modeled as singularly perturbed hybrid dynamical systems with resetting mechanisms are proposed, providing a unifying framework for hybrid and non-hybrid zero-order algorithms based on averaging theory.

Concurrent learning-based approximate optimal regulation

Accelerated Concurrent Learning Algorithms via Data-Driven Hybrid Dynamics and Nonsmooth ODEs

A novel class of accelerated data-driven concurrent learning algorithms suitable for the solution of high-performance system identification and parameter estimation problems with convergence certificates in settings where the standard persistence of excitation condition is difficult to guarantee or verify a priori.

Asymptotically Stable Adaptive–Optimal Control Algorithm With Saturating Actuators and Relaxed Persistence of Excitation

This paper proposes a control algorithm based on adaptive dynamic programming to solve the infinite-horizon optimal control problem for known deterministic nonlinear systems with saturating actuators…

Reinforcement Learning in Continuous Time and Space

  • K. Doya
  • Computer Science
    Neural Computation
  • 2000
This article presents a reinforcement learning framework for continuous-time dynamical systems without a priori discretization of time, state, and action. Basedonthe Hamilton-Jacobi-Bellman (HJB)…

Hybrid Dynamical Systems: Modeling, Stability, and Robustness

This book presents a complete theory of robust asymptotic stability for hybrid dynamical systems that is applicable to the design of hybrid control algorithms--algorithms that feature logic, timers, or combinations of digital and analog components.

The Heavy-Ball ODE with Time-Varying Damping: Persistence of Excitation and Uniform Asymptotic Stability

This work studies the uniform asymptotic stability properties of the heavy-ball optimization dynamics with general time-varying damping and shows that such conditions are related to the notion of persistence of excitation, which is commonly used in adaptive control and system identification.

Hybrid dynamical systems

Robust stability and control for systems that combine continuous-time and discrete-time dynamics. This article is a tutorial on modeling the dynamics of hybrid systems, on the elements of stability…

A Multi-Critic Reinforcement Learning Method: An Application to Multi-Tank Water Systems

This document highlights the benefits of the multi-critic methodology on a state of the art reinforcement learning algorithm, the deep deterministic policy gradient, and demonstrates its application to multi-tank water systems relevant for industrial process control.

Characterizations of input-to-state stability for hybrid systems