Sim2real for Autonomous Vehicle Control using Executable Digital Twin

  title={Sim2real for Autonomous Vehicle Control using Executable Digital Twin},
  author={Jean Pierre Allamaa and Panagiotis Patrinos and Herman Van der Auweraer and Tong Duy Son},
: In this work, we propose a sim2real method to transfer and adapt a nonlinear model predictive controller (NMPC) from simulation to the real target system based on executable digital twin (xDT). The xDT model is a high fidelity vehicle dynamics simulator, executable online in the control parameter randomization and learning process. The parameters are adapted to gradually improve control performance and deal with changing real-world environment. In particular, the performance metric is not… 

Figures from this paper

The Executable Digital Twin: merging the digital and the physics worlds

In this contribution, key building blocks such as model order reduction, real-time models, state estimation and co-simulation are reviewed, and a number of characteristic use cases are presented.




The essential feature of SPSA—which accounts for its power and relative ease of implementation—is the underlying gradient approximation that requires only two measurements of the objective function regardless of the dimension of the optimization problem.

A simulation-based testing and validation framework for ADAS development

This work focuses on simulation-based approaches to frontload control design verification during the early phases of ADAS development involving two software: LMS Imagine and PreScan, and yields a basis for the virtual validation and verification of autonomous driving functionalities.

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

This survey seeks to unify the field of AutoRL, provide a common taxonomy, discuss each area in detail and pose open problems of interest to researchers going forward.

Automated Controller Calibration by Kalman Filtering

The calibration method tunes the parameters online and robustly, is computationally efficient, has low data storage requirements, and is easy to implement making it appealing for many real-time applications.

Real-time Nonlinear MPC Strategy with Full Vehicle Validation for Autonomous Driving

The development and deployment of an embedded optimal control strategy for autonomous driving applications on a Ford Focus road vehicle is presented and the properties of sequential quadratic programming (SQP) optimization solvers that are suitable for driving tasks are shown.

A Probabilistic Graphical Model Foundation for Enabling Predictive Digital Twins at Scale

This work proposes a probabilistic graphical model as a formal mathematical representation of a digital twin and its associated physical asset as a set of coupled dynamical systems, evolving over time through their respective state-spaces and interacting via observed data and control inputs.

Digital Twins

The concept of Digital Twins and the chances for novel industrial applications are reviewed, and Mathematics are a key enabler and the impact will be highlighted along four specific examples addressing Digital Product Twins democratizing Design, Digital Production Twins enabling robots to mill, digital Production Twins driving industrialization of additive manufacturing, and Digital Performance Twins boosting operations.

Are We Making Real Progress in Simulated Environments? Measuring the Sim2Real Gap in Embodied Visual Navigation

The Habitat-PyRobot Bridge is developed, a library for seamless execution of identical code on a simulated agent and a physical robot and a new metric called Sim-vs-Real Correlation Coefficient (SRCC) is presented to quantify sim2real predictivity, which is largely due to AI agents learning to 'cheat' by exploiting simulator imperfections.

Assessing Transferability From Simulation to Reality for Reinforcement Learning

An algorithm called Simulation-based Policy Optimization with Transferability Assessment (SPOTA) which uses an estimator of the SOB to formulate a stopping criterion for training and is able to learn a control policy exclusively from a randomized simulator, which can be applied directly to real systems without any additional training.