On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements

@article{Dong2021OnCT,
  title={On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements},
  author={Suchuan Dong and Jielin Yang},
  journal={ArXiv},
  year={2021},
  volume={abs/2110.14121}
}
  • Suchuan Dong, Jielin Yang
  • Published 27 October 2021
  • Physics, Computer Science, Mathematics
  • ArXiv
We consider the use of extreme learning machines (ELM) for computational partial differential equations (PDE). In ELM the hidden-layer coefficients in the neural network are assigned to random values generated on [−Rm, Rm] and fixed, where Rm is a user-provided constant, and the output-layer coefficients are trained by a linear or nonlinear least squares computation. We present a method for computing the optimal or near-optimal value of Rm based on the differential evolution algorithm. This… 
1 Citations
PHYSICS-INFORMED RANDOM PROJECTION NEURAL NETWORKS
We propose a numerical method based on physics-informed Random Projection Neural Networks for the solution of Initial Value Problems (IVPs) of Ordinary Differential Equations (ODEs) with a focus on

References

SHOWING 1-10 OF 64 REFERENCES
Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations
  • S. Dong, Zongwei Li
  • Computer Science, Mathematics
    Computer Methods in Applied Mechanics and Engineering
  • 2021
TLDR
A neural network-based method for solving linear and nonlinear partial differential equations, by combining the ideas of extreme learning machines, domain decomposition and local neural networks, which exhibits a clear sense of convergence with respect to the degrees of freedom in the neural network.
Extreme learning machine collocation for the numerical solution of elliptic PDEs with sharp gradients
TLDR
It is shown that a feedforward neural network with a single hidden layer with sigmoidal functions and fixed, random, internal weights and biases can be used to compute accurately a collocation solution, thus avoiding the time-consuming training phase.
Numerical Solution and Bifurcation Analysis of Nonlinear Partial Differential Equations with Extreme Learning Machines
TLDR
It is shown that the proposed numerical machine learning method outperforms in terms of numerical accuracy both FD and FEM methods for medium to large sized grids, while provides equivalent results with the FEM for low to medium sized grids.
An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
TLDR
This contribution focuses in mechanical problems and analyze the energetic format of the PDE, where the energy of a mechanical system seems to be the natural loss function for a machine learning method to approach a mechanical problem.
Physics Informed Extreme Learning Machine (PIELM) - A rapid method for the numerical solution of partial differential equations
TLDR
This work develops and demonstrates that PIELM matches or exceeds the accuracy of PINNs on a range of problems, and shows that DPIELM produces excellent results comparable to conventional numerical techniques in the solution of time-dependent problems.
Artificial neural networks for solving ordinary and partial differential equations
TLDR
This article illustrates the method by solving a variety of model problems and presents comparisons with solutions obtained using the Galekrkin finite element method for several cases of partial differential equations.
The numerical solution of linear ordinary differential equations by feedforward neural networks
It is demonstrated, through theory and examples, how it is possible to construct directly and noniteratively a feedforward neural network to approximate arbitrary linear ordinary differential
Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
TLDR
A Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by incorporating the variational form of the problem into the loss function of the network and constructing a VPINN, effectively reducing the training cost in VPINNs while increasing their accuracy compared to PINNs that essentially employ delta test functions.
Solution of nonlinear ordinary differential equations by feedforward neural networks
It is demonstrated, through theory and numerical examples, how it is possible to directly construct a feedforward neural network to approximate nonlinear ordinary differential equations without the
A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines
TLDR
A modified batch intrinsic plasticity (modBIP) method for pre-training the random coefficients in the ELM neural networks, which is markedly more accurate than ELM/BIP in numerical simulations and insensitive to the random-coefficient initializations in the neural network.
...
1
2
3
4
5
...