On Controller Tuning with Time-Varying Bayesian Optimization

  title={On Controller Tuning with Time-Varying Bayesian Optimization},
  author={Paul Brunzema and Alexander von Rohr and Sebastian Trimpe},
  journal={2022 IEEE 61st Conference on Decision and Control (CDC)},
Changing conditions or environments can cause system dynamics to vary over time. To ensure optimal control performance, controllers should adapt to these changes. When the underlying cause and time of change is unknown, we need to rely on online data for this adaptation. In this paper, we will use time-varying Bayesian optimization (TVBO) to tune controllers online in changing environments using appropriate prior knowledge on the control objective and its changes. Two properties are… 

Figures and Tables from this paper

Improving the Performance of Robust Control through Event-Triggered Learning

This work proposes an event-triggered learning algorithm that decides when to learn in the face of uncertainty in the LQR problem with rare or slow changes, and designs a statistical test for uncertain systems based on the moment-generating function of the L QR cost.

Event-Triggered Time-Varying Bayesian Optimization

This work proposes an event-triggered algorithm, ET-GP-UCB, that detects changes in the objective function online and is competitive with state-of-the-art algorithms even though it requires no knowledge about the temporal changes.

Benchmark of Bayesian Optimization and Metaheuristics for Control Engineering Tuning Problems with Crash Constraints

—Controller tuning based on black-box optimization allows to automatically tune performance-critical parameters w.r.t. mostly arbitrary high-level closed-loop control objectives. However, a



Autonomous Vehicle Control Through the Dynamics and Controller Learning

This paper provides a novel algorithm, named as time-varying controller optimization, which takes both of the dynamic model uncertainty and the controller parameters uncertainty into account, and tunes them to find a global optimal choice for minimizing the time-Varying control costs.

Safe and Efficient Model-free Adaptive Control via Bayesian Optimization

This work proposes a purely data-driven, model-free approach for adaptive control that builds on GOOSE, an algorithm for safe and sample-efficient Bayesian optimization, and numerically demonstrates that this approach is sample efficient, outperforms constrained Bayesian optimize in terms of safety, and achieves the performance optima computed by grid evaluation.

On the design of LQR kernels for efficient controller learning

This work constructs two kernels that specifically leverage the structure of the well-known Linear Quadratic Regulator (LQR), yet retain the flexibility of Bayesian nonparametric learning.

Automatic LQR tuning based on Gaussian process global optimization

An automatic controller tuning framework based on linear optimal control combined with Bayesian optimization that shall yield improved controllers with fewer evaluations compared to alternative approaches is proposed.

Safe Bayesian Optimisation for Controller Design by Utilising the Parameter Space Approach

Analysis knowledge about robustly stable parameter configurations is gained by the parameter space approach and then incorporated within the optimisation as constraint and results show a significant performance gain compared to standard approaches.

Local policy search with Bayesian optimization

An algorithm utilizing a probabilistic model of the objective function and its gradient and based on the model, the algorithm decides where to query a noisy zeroth-order oracle to improve the gradient estimates, which reveals improved sample complexity and reduced variance in extensive empirical evaluations on synthetic objectives.

Kernel Recursive Least-Squares Tracker for Time-Varying Regression

This paper derives the standard KRLS equations from a Bayesian perspective and takes advantage of this framework to incorporate forgetting in a consistent way, thus enabling the algorithm to perform tracking in nonstationary scenarios and is the first kernel adaptive filtering algorithm that includes a forgetting factor in a principled and numerically stable manner.

Time-Varying Convex Optimization: Time-Structured Algorithms and Applications

A broad class of state-of-the-art algorithms for time-varying optimization is reviewed, with an eye to performing both algorithmic development and performance analysis, to exemplify wide engineering relevance of analytical tools and pertinent theoretical foundations.

Adapting to a Changing Environment: the Brownian Restless Bandits

The goal here is to characterize the cost of learning and adapting to the changing environment, in terms of the stochastic rate of the change, which is an infinite time horizon and defined with respect to a hypothetical algorithm that at every step plays the arm with the maximum expected reward at this step.

Machine learning meets Kalman Filtering

This work proposes a systematic and explicit procedure to address the problem of efficient non-parametric estimation for non-linear time-space dynamic Gaussian processes by pairing GP regression with Kalman Filtering and shows how to build an exact finite dimensional discrete-time state-space representation for the modeled process.