# Combining Physics and Deep Learning to learn Continuous-Time Dynamics Models

@article{Lutter2021CombiningPA, title={Combining Physics and Deep Learning to learn Continuous-Time Dynamics Models}, author={Michael Lutter and Jan Peters}, journal={ArXiv}, year={2021}, volume={abs/2110.01894} }

Deep learning has been widely used within learning algorithms for robotics. One disadvantage of deep networks is that these networks are black-box representations. Therefore, the learned approximations ignore the existing knowledge of physics or robotics. Especially for learning dynamics models, these black-box models are not desirable as the underlying principles are well understood and the standard deep networks can learn dynamics that violate these principles. To learn dynamics models with…

## 3 Citations

### Robots

- Computer ScienceEncyclopedia of Security and Emergency Management
- 2020

A Bayesian version of the Recursive Newton Euler Algorithm (DiffNEA) is investigated, able to predict the uncertainty of the model’s prediction and can be applied for techniques such as domain randomization to train robust policies.

### KeyCLD: Learning Constrained Lagrangian Dynamics in Keypoint Coordinates from Images

- Computer ScienceArXiv
- 2022

This work is the first to demonstrate learning of Lagrangian dynamics from images on the dm_control pendulum, cartpole and acrobot environments, and explicitly models kinetic and potential energy, thus allowing energy based control.

### A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic

- Computer ScienceArXiv
- 2022

A survey of ML applications in SD&V analyses, shedding light on the current state of implementation and emerging opportunities and considers the role of Digital Twins and Physics Guided ML to overcome current challenges and power future research progress.

## References

SHOWING 1-10 OF 93 REFERENCES

### Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning

- Computer ScienceICLR
- 2019

The proposed DeLaN network can learn the equations of motion of a mechanical system with a deep network efficiently while ensuring physical plausibility and exhibits substantially improved and more robust extrapolation to novel trajectories and learns online in real-time.

### Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning

- Physics2021 IEEE International Conference on Robotics and Automation (ICRA)
- 2021

This work demonstrates experimentally that for the offline model-based reinforcement learning setting, physics-based models can be beneficial compared to high-capacity function approximators if the mechanical structure is known and generalizes the approach of physics parameter identification from modeling holonomic multi-body systems to systems with nonholonomic dynamics using end-to-end automatic differentiation.

### Learning Contact Dynamics using Physically Structured Neural Networks

- Computer ScienceAISTATS
- 2021

The results indicate that an idealised form of touch feedback is a key component of making this learning problem tractable, and together with the inductive biases introduced through the network architectures, enable accurate learning of contact dynamics from observations.

### A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS

- Computer ScienceFront. Neurorobot.
- 2017

This paper proposes an implementation of a modern physics engine, which can differentiate control parameters, which is implemented for both CPU and GPU, and shows how such an engine speeds up the optimization process, even for small problems.

### Which priors matter? Benchmarking models for learning latent dynamics

- Computer ScienceNeurIPS Datasets and Benchmarks
- 2021

This work introduces a suite of 17 datasets with visual observations based on physical systems exhibiting a wide range of dynamics, and finds that the use of continuous and time-reversible dynamics benefits models of all classes.

### Encoding Physical Constraints in Differentiable Newton-Euler Algorithm

- Computer ScienceL4DC
- 2020

This work incorporates physical constraints in the learning by adding structure to the parameters of the differentiable RNEA algorithm, resulting in a framework that can learn physically plausible dynamics via gradient descent, improving the training speed as well as generalization of the learned dynamics models.

### Deep Lagrangian Networks for end-to-end learning of energy-based control for under-actuated systems

- Computer Science2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- 2019

The resulting DeLaN for energy control (DeLaN 4EC) is the first model learning approach using generic function approximation that is capable of learning energy control because existing approaches cannot learn the system energies directly.

### Learning Latent Dynamics for Planning from Pixels

- Computer ScienceICML
- 2019

The Deep Planning Network (PlaNet) is proposed, a purely model-based agent that learns the environment dynamics from images and chooses actions through fast online planning in latent space using a latent dynamics model with both deterministic and stochastic transition components.

### Using model knowledge for learning inverse dynamics

- Computer Science2010 IEEE International Conference on Robotics and Automation
- 2010

This paper investigates how parametric physical models (e.g., obtained from rigid body dynamics) can be used to improve the learning performance, and how semiparametric regression methods can be applied in this context.

### A Differentiable Newton Euler Algorithm for Multi-body Model Learning

- Computer ScienceArXiv
- 2020

The main contributions of this work are the introduction of a white-box model that jointly learns dynamic and kinematics parameters and can be combined with black-box components.