• Corpus ID: 237304254

Unsupervised Reservoir Computing for Solving Ordinary Differential Equations

@article{Mattheakis2021UnsupervisedRC,
  title={Unsupervised Reservoir Computing for Solving Ordinary Differential Equations},
  author={Marios Mattheakis and Hayden Joy and Pavlos Protopapas},
  journal={ArXiv},
  year={2021},
  volume={abs/2108.11417}
}
There is a wave of interest in using unsupervised neural networks for solving differential equations. The existing methods are based on feed-forward networks, while recurrent neural network differential equation solvers have not yet been reported. We introduce an unsupervised reservoir computing (RC), an echo-state recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs). We suggest an approach to calculate time derivatives of… 

Figures and Tables from this paper

RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimization
TLDR
The utility of RcTorch, a PyTorch based RC neural network package with automated HP tuning is demonstrated by using it to predict the complex dynamics of a driven pendulum being acted upon by varying forces.
Physics-Informed Neural Networks for Quantum Eigenvalue Problems
TLDR
This proposed unsupervised learning method is used to solve the finite well, multiple finite wells, and hydrogen atom eigenvalue quantum problems and finds that embedding even or odd symmetries to the neural network architecture further improves the convergence for relevant problems.
A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data
TLDR
A data-driven approach to automatically extract an appropriate model structure from experimentally observed “related” processes that can be used to vastly reduce the amount of data required to successfully train a predictive model is presented.
Hyperverlet: A Symplectic Hypersolver for Hamiltonian Systems
TLDR
Hyperverlet is introduced, a new hypersolver composing the traditional, symplectic velocity Verlet and symplectic neural network-based solvers and it is proved that hyperbolic tangent is r-finite expanding the set of allowable activation functions for symp eclectic neural networks, improving the accuracy.
Uncertainty Quantification in Neural Differential Equations
TLDR
This work adapts several state-of-the-art UQ methods to get the predictive uncertainty for DE solutions and shows the results on four different DE types.

References

SHOWING 1-10 OF 44 REFERENCES
Solving Differential Equations Using Neural Network Solution Bundles
TLDR
A neural network is proposed to be used as a solution bundle, a collection of solutions to an ODE for various initial states and system parameters, which exhibits fast, parallelizable evaluation of the system state, facilitating the use of Bayesian inference for parameter estimation in real dynamical systems.
Artificial neural networks for solving ordinary and partial differential equations
TLDR
This article illustrates the method by solving a variety of model problems and presents comparisons with solutions obtained using the Galekrkin finite element method for several cases of partial differential equations.
Hamiltonian Neural Networks for solving differential equations
TLDR
A Hamiltonian neural network that solves the differential equations that govern dynamical systems and is considered a symplectic unit due to the introduction of an efficient parametric form of solutions.
Neural Ordinary Differential Equations
TLDR
This work shows how to scalably backpropagate through any ODE solver, without access to its internal operations, which allows end-to-end training of ODEs within larger models.
NeuroDiffEq: A Python package for solving differential equations with neural networks
TLDR
Time integration techniques continue to be an active area of research and include backward difference formulas and Runge-Kutta methods and common spatial discretization approaches include the finite difference method, finite volume method, and finite element method.
Reservoir Computing meets Recurrent Kernels and Structured Transforms
TLDR
The rigorously establish the recurrent kernel limit of Reservoir Computing and prove its convergence, and shows how the Recurrent Kernel is competitive and computationally efficient when the number of data points remains moderate.
Solving high-dimensional partial differential equations using deep learning
TLDR
A deep learning-based approach that can handle general high-dimensional parabolic PDEs using backward stochastic differential equations and the gradient of the unknown solution is approximated by neural networks, very much in the spirit of deep reinforcement learning with the gradient acting as the policy function.
Data-driven discovery of partial differential equations
TLDR
The sparse regression method is computationally efficient, robust, and demonstrated to work on a variety of canonical problems spanning a number of scientific domains including Navier-Stokes, the quantum harmonic oscillator, and the diffusion equation.
...
...