# The Random Feature Model for Input-Output Maps between Banach Spaces

@article{Nelsen2021TheRF, title={The Random Feature Model for Input-Output Maps between Banach Spaces}, author={Nicholas H. Nelsen and Andrew M. Stuart}, journal={ArXiv}, year={2021}, volume={abs/2005.10224} }

Well known to the machine learning community, the random feature model, originally introduced by Rahimi and Recht in 2008, is a parametric approximation to kernel interpolation or regression methods. It is typically used to approximate functions mapping a finite-dimensional input space to the real line. In this paper, we instead propose a methodology for use of the random feature model as a data-driven surrogate for operators that map an input Banach space to an output Banach space. Although… Expand

#### Figures and Tables from this paper

#### 27 Citations

Deep Decomposition for Stochastic Normal-Abnormal Transport

- Computer Science
- ArXiv
- 2021

Advection-diffusion equations describe a large family of natural transport processes, e.g., fluid flow, heat transfer, and wind transport. They are also used for optical flow and perfusion imaging… Expand

Neural Operator: Learning Maps Between Function Spaces

- Computer Science, Mathematics
- ArXiv
- 2021

A generalization of neural networks tailored to learn operators mapping between infinite dimensional function spaces, formulated by composition of a class of linear integral operators and nonlinear activation functions, so that the composed operator can approximate complex nonlinear operators. Expand

A generalization of the randomized singular value decomposition

- Computer Science, Mathematics
- ArXiv
- 2021

This work generalizes the theory of randomized SVD to multivariable Gaussian vectors, allowing one to incorporate prior knowledge of A into the algorithm, and constructs a new covariance kernel for GPs, based on weighted Jacobi polynomials. Expand

Calibration and Uncertainty Quantification of Convective Parameters in an Idealized GCM

- 2021

Parameters in climate models are usually calibrated manually, exploiting only small subsets of the available data. This precludes an optimal calibration and quantification of uncertainties.… Expand

Choose a Transformer: Fourier or Galerkin

- Computer Science, Mathematics
- ArXiv
- 2021

It is demonstrated for the first time that the softmax normalization in the scaled dot-product attention is sufficient but not necessary and the newly proposed simple attention-based operator learner, Galerkin Transformer shows significant improvements in both training cost and evaluation accuracy over its softmax-normalized counterparts. Expand

Convective Parameters in an Idealized GCM

- 2021

14 Parameters in climate models are usually calibrated manually, exploiting only small sub15 sets of the available data. This precludes both optimal calibration and quantification of 16… Expand

Convergence Rates for Learning Linear Operators from Noisy Data

- Computer Science, Mathematics
- ArXiv
- 2021

This work establishes posterior contraction rates with respect to a family of Bochner norms as the number of data tend to infinity and derive related lower bounds on the estimation error and connects the posterior consistency results to nonparametric learning theory. Expand

Discovering Hidden Physics Behind Transport Dynamics

- Computer Science, Engineering
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2021

A learning framework (YETI) building on an auto-encoder structure between 2D and 3D image time-series, which incorporates the advection-diffusion model and introduces representations that assure incompressible flow and symmetric positive semi-definite diffusion fields and demonstrates the additional benefits of these representations on improving estimation accuracy. Expand

Fourier Neural Operator for Parametric Partial Differential Equations

- Computer Science, Mathematics
- ICLR
- 2021

This work forms a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture and shows state-of-the-art performance compared to existing neural network methodologies. Expand

Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model

- Computer Science, Mathematics
- ArXiv
- 2021

This paper takes advantages of the low-rank covariance structure to reduce the number of forward problem evaluations and the memory cost, related to the need to propagate large covariance matrices, and uses reduced-order model techniques to further speed up these forward evaluations. Expand

#### References

SHOWING 1-10 OF 122 REFERENCES

Operator-valued kernels for learning from functional response data

- Mathematics
- 2016

In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label ...

Random Features for Large-Scale Kernel Machines

- Computer Science, Mathematics
- NIPS
- 2007

Two sets of random features are explored, provided convergence bounds on their ability to approximate various radial basis kernels, and it is shown that in large-scale classification and regression tasks linear machine learning algorithms applied to these features outperform state-of-the-art large- scale kernel machines. Expand

On the Approximation Properties of Random ReLU Features

- Mathematics
- 2018

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features… Expand

On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2017

Theoretical analysis of the number of required samples for a given approximation error leads to both upper and lower bounds that are based solely on the eigenvalues of the associated integral operator and match up to logarithmic terms. Expand

Operator-valued Kernels for Learning from Functional Response Data

- Computer Science, Mathematics
- J. Mach. Learn. Res.
- 2016

In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label is… Expand

Approximation of high-dimensional parametric PDEs *

- Mathematics, Computer Science
- Acta Numerica
- 2015

Results reveal that, at least on a theoretical level, the solution map can be well approximated by discretizations of moderate complexity, thereby showing how the curse of dimensionality is broken. Expand

Uniform approximation of functions with random bases

- Mathematics
- 2008 46th Annual Allerton Conference on Communication, Control, and Computing
- 2008

Random networks of nonlinear functions have a long history of empirical success in function fitting but few theoretical guarantees. In this paper, using techniques from probability on Banach Spaces,… Expand

Priors for Infinite Networks

- Mathematics
- 1996

In this chapter, I show that priors over network parameters can be defined in such a way that the corresponding priors over functions computed by the network reach reasonable limits as the number of… Expand

Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs

- Mathematics, Computer Science
- Computer Methods in Applied Mechanics and Engineering
- 2022

This work proposes to construct surrogates for high-dimensional PDE-governed parametric maps in the form of projected neural networks that parsimoniously capture the geometry and intrinsic low-dimensionality of these maps. Expand

Fourier Neural Operator for Parametric Partial Differential Equations

- Computer Science, Mathematics
- ICLR
- 2021

This work forms a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture and shows state-of-the-art performance compared to existing neural network methodologies. Expand