Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model

  title={Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model},
  author={Jaideep Pathak and Alexander Wikner and Rebeckah Fussell and Sarthak Chandra and Brian R. Hunt and Michelle Girvan and Edward Ott},
  volume={28 4},
A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the mechanistic processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time series measurements of system state variables (training data), without prior knowledge of the system dynamics. The motivation for this paper is the potential of machine… 

Figures and Tables from this paper

Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems

It is demonstrated that not only can this method combining (i) and (ii) be scaled to give excellent performance for very large systems but also that the length of time series data needed to train the authors' multiple, parallel machine learning components is dramatically less than that necessary without parallelization.

Forecasting macroscopic dynamics in adaptive Kuramoto network using reservoir computing.

This work trained reservoir computer to predict the macroscopic signal produced by the network of phase oscillators and found that short-time prediction required a large number of features, while long- time prediction utilizes a limited number of Features.

Applying Machine Learning to Improve Simulations of a Chaotic Dynamical System Using Empirical Error Correction

  • P. Watson
  • Environmental Science
    Journal of advances in modeling earth systems
  • 2019
A framework using machine learning together with physically‐derived models is tested, and it is shown that the approach yields models that are stable and that give both improved skill in initialized predictions and better long‐term climate statistics.

Optimal reservoir computers for forecasting systems of nonlinear dynamics

It is shown that reservoirs of low connectivity perform better than or as well as those of high connectivity in forecasting noiseless Lorenz and coupled Wilson-Cowan systems, and that, unexpectedly, computationally effective reservoirs of unconnected nodes (RUN) outperform reservoirs of linked network topologies in predicting these systems.

Reservoir Computing with Diverse Timescales for Prediction of Multiscale Dynamics

A reservoir computing model with diverse timescales by using a recurrent network of heterogeneous leaky integrator (LI) neurons is proposed, and it is demonstrated that a closed-loop version of the proposed model can achieve longer-term predictions compared to the counterpart with identical LI neurons depending on the hyperparameter setting.

Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks: a comparative study of Reservoir Computing and Backpropagation Algorithms

This study confirms that RNNs present a potent computational framework for the forecasting of complex spatio-temporal dynamics.

Short- and long-term predictions of chaotic flows and extreme events: a physics-constrained reservoir computing approach

We propose a physics-constrained machine learning method—based on reservoir computing—to time-accurately predict extreme events and long-term velocity statistics in a model of chaotic flow. The

Attractor Ranked Radial Basis Function Network: A Nonparametric Forecasting Approach for Chaotic Dynamic Systems

This work presents a forecasting algorithm that exploits the dimensionality of data in a nonparametric autoregressive framework and shows that the approach, attractor ranked radial basis function network (AR-RBFN) provides a better forecast than that obtained using other model-free approaches as well as univariate and multivariate autore progressive models using radial basisfunction networks.



Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data.

This work uses recent advances in the machine learning area known as "reservoir computing" to formulate a method for model-free estimation from data of the Lyapunov exponents of a chaotic process to form a modified autonomous reservoir.

Reservoir observers: Model-free inference of unmeasured variables in chaotic systems.

It is shown that the reservoir observer can be a very effective and versatile tool for robustly reconstructing unmeasured dynamical system variables.

Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach.

We demonstrate the effectiveness of using machine learning for model-free prediction of spatiotemporally chaotic systems of arbitrarily large spatial extent and attractor dimension purely from

A hybrid neural network‐first principles approach to process modeling

A hybrid neural network-first principles modeling scheme is developed and used to model a fedbatch bioreactor. The hybrid model combines a partial first principles model, which incorporates the

Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations

A new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks, based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry.

Reservoir computing with a single time-delay autonomous Boolean node

We demonstrate reservoir computing with a physical system using a single autonomous Boolean logic element with time-delay feedback. The system generates a chaotic transient with a window of

Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication

We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning

Brain-Inspired Photonic Signal Processor for Generating Periodic Patterns and Emulating Chaotic Systems

This work presents a photonic reservoir computer with output feedback, and demonstrates its capacity to generate periodic time series and to emulate chaotic systems, and introduces several metrics, based on standard signal processing techniques, to evaluate the quality of the emulation.

Long Short-Term Memory

A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.