# Interactive Differentiable Simulation

@article{Heiden2019InteractiveDS, title={Interactive Differentiable Simulation}, author={Eric Heiden and David Millard and Hejia Zhang and Gaurav S. Sukhatme}, journal={ArXiv}, year={2019}, volume={abs/1905.10706} }

Intelligent agents need a physical understanding of the world to predict the impact of their actions in the future. While learning-based models of the environment dynamics have contributed to significant improvements in sample efficiency compared to model-free reinforcement learning algorithms, they typically fail to generalize to system states beyond the training data, while often grounding their predictions on non-interpretable latent variables.
We introduce Interactive Differentiable…

## 31 Citations

Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning

- Physics2021 IEEE International Conference on Robotics and Automation (ICRA)
- 2021

This work demonstrates experimentally that for the offline model-based reinforcement learning setting, physics-based models can be beneficial compared to high-capacity function approximators if the mechanical structure is known and generalizes the approach of physics parameter identification from modeling holonomic multi-body systems to systems with nonholonomic dynamics using end-to-end automatic differentiation.

NeuralSim: Augmenting Differentiable Simulators with Neural Networks

- Computer Science2021 IEEE International Conference on Robotics and Automation (ICRA)
- 2021

Through extensive experiments, the ability of the hybrid simulator to learn complex dynamics involving frictional contacts from real data, as well as match known models of viscous friction, is demonstrated, and an approach for automatically discovering useful augmentations is presented.

Iterative residual tuning for system identification and sim-to-real robot learning

- Computer ScienceAuton. Robots
- 2020

This paper presents iterative residual tuning (IRT), a deep learning system identification technique that modifies a simulator’s parameters to better match reality using minimal real-world observations, and develops and analyzes IRT in depth.

Sim2Sim Evaluation of a Novel Data-Efficient Differentiable Physics Engine for Tensegrity Robots

- Computer Science2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- 2021

This work proposes a data-driven, end-to-end differentiable simulator focused on the exciting but challenging domain of tensegrity robots, and demonstrates sim2sim transfer, where the unknown physical model of MuJoCo acts as a ground truth system.

PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics

- Computer ScienceICLR
- 2021

A new differentiable physics benchmark called PasticineLab is introduced, which includes a diverse collection of soft body manipulation tasks, and experimental results suggest that RL-based approaches struggle to solve most of the tasks efficiently and gradient- based approaches can rapidly find a solution within tens of iterations, but still fall short on multi-stage tasks that require long-term planning.

Real2Sim Transfer using Differentiable Physics

- Computer Science
- 2019

This work introduces a new layer in the deep learning toolbox that imposes a strong inductive bias to generate physically accurate predictions of rigid-body dynamics and allows for the automatic inference of system parameters given an ad-hoc model description.

Sparse-Input Neural Network Augmentations for Differentiable Simulators

- Computer Science
- 2020

This work studies the augmentation of a differentiable rigid-body physics engine via neural networks that is able to learn nonlinear relationships between dynamic quantities and can thus learn effects not accounted for in the traditional simulator.

A Differentiable Newton-Euler Algorithm for Real-World Robotics

- Computer ScienceArXiv
- 2021

A differentiable simulator that can be used to identify the system parameters of real-world mechanical systems with complex friction models, holonomic as well as non-holonomic constraints and can be applied to a class of dynamical systems and guarantees physically plausible predictions.

Closing the Sim2Real Gap using Invertible Simulators

- Computer Science
- 2020

This work proposes to design the simulator from the ground up to be invertible, i.e., such that it can estimate the simulation settings from the observations of the real system, and approach the problem of visuomotor control from a different angle.

gradSim: Differentiable simulation for system identification and visuomotor control

- Computer ScienceICLR
- 2021

We consider the problem of estimating an object’s physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally…

## References

SHOWING 1-10 OF 35 REFERENCES

Continuous control with deep reinforcement learning

- Computer ScienceICLR
- 2016

This work presents an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces, and demonstrates that for many of the tasks the algorithm can learn policies end-to-end: directly from raw pixel inputs.

End-to-End Differentiable Physics for Learning and Control

- Computer ScienceNeurIPS
- 2018

This paper demonstrates how to perform backpropagation analytically through a physical simulator defined via a linear complementarity problem, and highlights the system's ability to learn physical parameters from data, efficiently match and simulate observed visual behavior, and readily enable control via gradient-based planning methods.

Graph networks as learnable physics engines for inference and control

- Computer ScienceICML
- 2018

A new class of learnable models are introduced--based on graph networks--which implement an inductive bias for object- and relation-centric representations of complex, dynamical systems, and offers new opportunities for harnessing and exploiting rich knowledge about the world.

Flexible Neural Representation for Physics Prediction

- PhysicsNeurIPS
- 2018

The Hierarchical Relation Network (HRN) is described, an end-to-end differentiable neural network based on hierarchical graph convolution that learns to predict physical dynamics in this hierarchical particle-based object representation.

Fast Model Identification via Physics Engines for Data-Efficient Policy Search

- Computer ScienceIJCAI
- 2018

Evaluations indicate that the proposed strategy for identifying a model that is just good enough to approximate the value of a locally optimal policy with a certain confidence results in an overall time-efficient, integrated model identification and learning solution, which significantly improves the data-efficiency of existing policy search algorithms.

Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning

- Computer ScienceICLR
- 2019

The proposed DeLaN network can learn the equations of motion of a mechanical system with a deep network efficiently while ensuring physical plausibility and exhibits substantially improved and more robust extrapolation to novel trajectories and learns online in real-time.

Unsupervised Learning for Physical Interaction through Video Prediction

- Computer ScienceNIPS
- 2016

An action-conditioned video prediction model is developed that explicitly models pixel motion, by predicting a distribution over pixel motion from previous frames, and is partially invariant to object appearance, enabling it to generalize to previously unseen objects.

Neural networks and differential dynamic programming for reinforcement learning problems

- Computer Science2016 IEEE International Conference on Robotics and Automation (ICRA)
- 2016

This paper extends neural networks for modeling prediction error and output noise, computing an output probability distribution for a given input distribution, and computing gradients of output expectation with respect to an input, and provides an analytic solution for these extensions.

Survey of Model-Based Reinforcement Learning: Applications on Robotics

- Computer ScienceJ. Intell. Robotic Syst.
- 2017

It is argued that, by employing model-based reinforcement learning, the—now limited—adaptability characteristics of robotic systems can be expanded, and model- based reinforcement learning exhibits advantages that makes it more applicable to real life use-cases compared to model-free methods.

Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

- Computer ScienceICLR
- 2019

This study helps lay the foundation for robot learning of dynamic scenes with particle-based representations, and demonstrates robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam.