The Random Feature Model for Input-Output Maps between Banach Spaces

@article{Nelsen2021TheRF,
  title={The Random Feature Model for Input-Output Maps between Banach Spaces},
  author={Nicholas H. Nelsen and Andrew M. Stuart},
  journal={ArXiv},
  year={2021},
  volume={abs/2005.10224}
}
Well known to the machine learning community, the random feature model, originally introduced by Rahimi and Recht in 2008, is a parametric approximation to kernel interpolation or regression methods. It is typically used to approximate functions mapping a finite-dimensional input space to the real line. In this paper, we instead propose a methodology for use of the random feature model as a data-driven surrogate for operators that map an input Banach space to an output Banach space. Although… Expand

Figures and Tables from this paper

Deep Decomposition for Stochastic Normal-Abnormal Transport
Advection-diffusion equations describe a large family of natural transport processes, e.g., fluid flow, heat transfer, and wind transport. They are also used for optical flow and perfusion imagingExpand
Neural Operator: Learning Maps Between Function Spaces
TLDR
A generalization of neural networks tailored to learn operators mapping between infinite dimensional function spaces, formulated by composition of a class of linear integral operators and nonlinear activation functions, so that the composed operator can approximate complex nonlinear operators. Expand
A generalization of the randomized singular value decomposition
TLDR
This work generalizes the theory of randomized SVD to multivariable Gaussian vectors, allowing one to incorporate prior knowledge of A into the algorithm, and constructs a new covariance kernel for GPs, based on weighted Jacobi polynomials. Expand
Calibration and Uncertainty Quantification of Convective Parameters in an Idealized GCM
Parameters in climate models are usually calibrated manually, exploiting only small subsets of the available data. This precludes an optimal calibration and quantification of uncertainties.Expand
Choose a Transformer: Fourier or Galerkin
TLDR
It is demonstrated for the first time that the softmax normalization in the scaled dot-product attention is sufficient but not necessary and the newly proposed simple attention-based operator learner, Galerkin Transformer shows significant improvements in both training cost and evaluation accuracy over its softmax-normalized counterparts. Expand
Convective Parameters in an Idealized GCM
14 Parameters in climate models are usually calibrated manually, exploiting only small sub15 sets of the available data. This precludes both optimal calibration and quantification of 16Expand
Convergence Rates for Learning Linear Operators from Noisy Data
TLDR
This work establishes posterior contraction rates with respect to a family of Bochner norms as the number of data tend to infinity and derive related lower bounds on the estimation error and connects the posterior consistency results to nonparametric learning theory. Expand
Discovering Hidden Physics Behind Transport Dynamics
TLDR
A learning framework (YETI) building on an auto-encoder structure between 2D and 3D image time-series, which incorporates the advection-diffusion model and introduces representations that assure incompressible flow and symmetric positive semi-definite diffusion fields and demonstrates the additional benefits of these representations on improving estimation accuracy. Expand
Fourier Neural Operator for Parametric Partial Differential Equations
TLDR
This work forms a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture and shows state-of-the-art performance compared to existing neural network methodologies. Expand
Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model
TLDR
This paper takes advantages of the low-rank covariance structure to reduce the number of forward problem evaluations and the memory cost, related to the need to propagate large covariance matrices, and uses reduced-order model techniques to further speed up these forward evaluations. Expand
...
1
2
3
...

References

SHOWING 1-10 OF 122 REFERENCES
Operator-valued kernels for learning from functional response data
In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label ...
Random Features for Large-Scale Kernel Machines
TLDR
Two sets of random features are explored, provided convergence bounds on their ability to approximate various radial basis kernels, and it is shown that in large-scale classification and regression tasks linear machine learning algorithms applied to these features outperform state-of-the-art large- scale kernel machines. Expand
On the Approximation Properties of Random ReLU Features
We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random featuresExpand
On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions
  • F. Bach
  • Mathematics, Computer Science
  • J. Mach. Learn. Res.
  • 2017
TLDR
Theoretical analysis of the number of required samples for a given approximation error leads to both upper and lower bounds that are based solely on the eigenvalues of the associated integral operator and match up to logarithmic terms. Expand
Operator-valued Kernels for Learning from Functional Response Data
In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label isExpand
Approximation of high-dimensional parametric PDEs *
TLDR
Results reveal that, at least on a theoretical level, the solution map can be well approximated by discretizations of moderate complexity, thereby showing how the curse of dimensionality is broken. Expand
Uniform approximation of functions with random bases
  • A. Rahimi, B. Recht
  • Mathematics
  • 2008 46th Annual Allerton Conference on Communication, Control, and Computing
  • 2008
Random networks of nonlinear functions have a long history of empirical success in function fitting but few theoretical guarantees. In this paper, using techniques from probability on Banach Spaces,Expand
Priors for Infinite Networks
In this chapter, I show that priors over network parameters can be defined in such a way that the corresponding priors over functions computed by the network reach reasonable limits as the number ofExpand
Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs
TLDR
This work proposes to construct surrogates for high-dimensional PDE-governed parametric maps in the form of projected neural networks that parsimoniously capture the geometry and intrinsic low-dimensionality of these maps. Expand
Fourier Neural Operator for Parametric Partial Differential Equations
TLDR
This work forms a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture and shows state-of-the-art performance compared to existing neural network methodologies. Expand
...
1
2
3
4
5
...