On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements

  title={On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements},
  author={Suchuan Dong and Jielin Yang},
We consider the use of extreme learning machines (ELM) for computational partial differential equations (PDE). In ELM the hidden-layer coefficients in the neural network are assigned to random values generated on [−Rm, Rm] and fixed, where Rm is a user-provided constant, and the output-layer coefficients are trained by a linear or nonlinear least squares computation. We present a method for computing the optimal or near-optimal value of Rm based on the differential evolution algorithm. This… 
Numerical Approximation of Partial Differential Equations by a Variable Projection Method with Artificial Neural Networks
We present a method for solving linear and nonlinear partial differential equations (PDE) based on the variable projection framework and artificial neural networks. For linear PDEs, enforcing the
The proposed numerical method based on physics-informed Random Projection Neural Networks for the solution of Initial Value Problems (IVPs) of Ordinary Differential Equations (ODEs) with a focus on stiff problems yields good approximation accuracy, thus outperforming ode45 and ode15s.
Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine
This paper presents a modified ELM method, termed HLConcELM (hidden-layer concatenated ELM), based on a type of modified feedforward neural networks (FNN), which incorporates a logical concatenation of the hidden layers in the network and exposes all the hidden nodes to the output-layer nodes.


Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations
  • S. Dong, Zongwei Li
  • Computer Science
    Computer Methods in Applied Mechanics and Engineering
  • 2021
Extreme learning machine collocation for the numerical solution of elliptic PDEs with sharp gradients
Numerical Solution and Bifurcation Analysis of Nonlinear Partial Differential Equations with Extreme Learning Machines
It is shown that the proposed numerical machine learning method outperforms in terms of numerical accuracy both FD and FEM methods for medium to large sized grids, while provides equivalent results with the FEM for low to medium sized grids.
An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
Artificial neural networks for solving ordinary and partial differential equations
This article illustrates the method by solving a variety of model problems and presents comparisons with solutions obtained using the Galekrkin finite element method for several cases of partial differential equations.
Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
A Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by incorporating the variational form of the problem into the loss function of the network and constructing a VPINN, effectively reducing the training cost in VPINNs while increasing their accuracy compared to PINNs that essentially employ delta test functions.
Extreme learning machine: Theory and applications