Deep learning of dynamics and signal-noise decomposition with time-stepping constraints

  title={Deep learning of dynamics and signal-noise decomposition with time-stepping constraints},
  author={Samuel H. Rudy and J. Nathan Kutz and Steven L. Brunton},
  journal={J. Comput. Phys.},

Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint

The proposed approach provides a promising methodology to learn dynamic models, where the first-principle understanding remains opaque, by learning a neural network that implicitly represents the data and an additional network that models the vector fields of the dependent variables.

Neural ODEs with Irregular and Noisy Data

The proposed framework to learn a model describing the vector field is highly effective under noisy measurements and can handle scenarios where dependent variables are not available at the same temporal grid.

The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation

This work considers the deep network-based LMMs for the discovery of dynamics using the approximation property of deep networks, and indicates, for certain families of LMMs, that the l grid error is bounded by the sum of O(h) and the network approximation error.

Learning Fine Scale Dynamics from Coarse Observations via Inner Recurrence

  • V. ChurchillD. Xiu
  • Computer Science
    Journal of Machine Learning for Modeling and Computing
  • 2022
This paper presents a computational technique to learn thene-scale dynamics from such coarsely observed data and employs inner recurrence of a DNN to recover the ne-scale evolution operator of the underlying system.

Active operator inference for learning low-dimensional dynamical-system models from noisy data

This work builds on operator inference from scientific machine learning to infer low-dimensional models from high-dimensional state trajectories polluted with noise and shows that, under certain conditions, the inferred operators are unbiased estimators of the well-studied projection-based reduced operators from traditional model reduction.

Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Prediction

We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models in various gray-box settings which incorporates prior knowledge in the form of systems of ordinary differential

LQResNet: A Deep Neural Network Architecture for Learning Dynamic Processes

This work suggests combining the operator inference with certain deep neural network approaches to infer the unknown nonlinear dynamics of the system and demonstrates that the proposed methodology accomplishes the desired tasks for dynamics processes encountered in neural dynamics and the glycolytic oscillator.

Discovery of nonlinear dynamical systems using a Runge–Kutta inspired dictionary-based sparse regression approach

This work combines machine learning and dictionary-based learning with numerical analysis tools to discover differential equations from noisy and sparsely sampled measurement data of time-dependent processes, and extends the method to governing equations, containing rational nonlinearities that typically appear in biological networks.

Learning Low-Dimensional Quadratic-Embeddings of High-Fidelity Nonlinear Dynamics using Deep Learning

This work leverages deep learning to identify low-dimensional quadratic embeddings for high-fidelity dynamical systems and embeds a Runge-Kutta method to avoid the time-derivative computations.

Robust Modeling of Unknown Dynamical Systems via Ensemble Averaged Learning

This paper presents a computational technique which decreases the variance of the generalization error, thereby improving the reliability of the DNN model to generalize consistently and in the proposed ensemble averaging method.



Extracting structured dynamical systems using sparse optimization with very few samples

A random sampling method for learning structured dynamical systems from under-sampled and possibly noisy state-space measurements based on a Bernstein-like inequality for partly dependent random variables and theoretical guarantees on the recovery rate of the sparse coefficients and the identification of the candidate functions for the corresponding problem.

Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

  • M. Raissi
  • Computer Science
    J. Mach. Learn. Res.
  • 2018
This work puts forth a deep learning approach for discovering nonlinear partial differential equations from scattered and potentially noisy observations in space and time by approximate the unknown solution as well as the nonlinear dynamics by two deep neural networks.

Deep learning for universal linear embeddings of nonlinear dynamics

It is often advantageous to transform a strongly nonlinear system into a linear one in order to simplify its analysis for prediction and control, so the authors combine dynamical systems with deep learning to identify these hard-to-find transformations.

Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems

This work puts forth a machine learning approach for identifying nonlinear dynamical systems from data that combines classical tools from numerical analysis with powerful nonlinear function approximators to distill the mechanisms that govern the evolution of a given data-set.

Extracting Sparse High-Dimensional Dynamics from Limited Data

This work details three sampling strategies that lead to the exact recovery of first-order dynamical systems when the authors are given fewer samples than unknowns, which is stable to the sparse structure of the governing equations and robust to noise in the estimation of the velocity.

Data Driven Governing Equations Approximation Using Deep Neural Networks

Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems

This paper introduces a computational framework for learning Koopman operators of nonlinear dynamical systems using deep learning and shows that this novel method automatically selects efficient deep dictionaries, requiring much lower dimensional dictionaries while outperforming state-of-the-art methods.

Hidden physics models: Machine learning of nonlinear partial differential equations

Sparse reduced-order modelling: sensor-based dynamics to full-state estimation

A grey-box modelling strategy is successfully applied to the transient and post-transient laminar cylinder wake, and compares favourably with a proper orthogonal decomposition model.

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

This two part treatise introduces physics informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations and demonstrates how these networks can be used to infer solutions topartial differential equations, and obtain physics-informed surrogate models that are fully differentiable with respect to all input coordinates and free parameters.