# On The Relative Error of Random Fourier Features for Preserving Kernel Distance

@article{Cheng2022OnTR,
title={On The Relative Error of Random Fourier Features for Preserving Kernel Distance},
author={Kuan Cheng and Shaofeng H.-C. Jiang and Luojian Wei and Zhide Wei},
journal={ArXiv},
year={2022},
volume={abs/2210.00244}
}
• Published 1 October 2022
• Computer Science
• ArXiv
The method of random Fourier features (RFF), proposed in a seminal paper by Rahimi and Recht (NIPS’07), is a powerful technique to ﬁnd approximate low-dimensional representations of points in (high-dimensional) kernel space, for shift-invariant kernels. While RFF has been analyzed under various notions of error guarantee, the ability to preserve the kernel distance with relative error is less under-stood. We show that for a signiﬁcant range of kernels, including the well-known Laplacian kernels…

## References

SHOWING 1-10 OF 54 REFERENCES

• Computer Science
ALT
• 2017
It is shown in this paper that for the Gaussian kernel the Euclidean norm between these mapped to features has \$(1+\epsilon)-relative error with respect to the kernel distance.
• Computer Science
NIPS
• 2015
A detailed finite-sample theoretical analysis about the approximation quality of RFFs is provided by establishing optimal (in terms of the RFF dimension, and growing set size) performance guarantees in uniform norm, and presenting guarantees in Lr (1 ≤ r < ∞) norms.
• Computer Science
UAI
• 2015
The uniform error bound of that paper on random Fourier features is improved, as well as giving novel understandings of the embedding's variance, approximation error, and use in some machine learning methods.
• Computer Science
ICML
• 2019
This work provides the first unified risk analysis of learning with random Fourier features using the squared error and Lipschitz continuous loss functions and devise a simple approximation scheme which provably reduces the computational cost without loss of statistical efficiency.
• Computer Science
ICML
• 2020
A near input sparsity time algorithm for sampling the high-dimensional feature space implicitly defined by a kernel transformation, and shows how its subspace embedding bounds imply new statistical guarantees for kernel ridge regression.
• Computer Science, Mathematics
AISTATS
• 2018
It is proved that for the case of many dimensions, the superiority of the orthogonal transform can be accurately measured by a property called the charm of the kernel, and that Orthogonal random features provide optimal (in terms of mean squared error) kernel estimators.
• Computer Science
AISTATS
• 2019
This work proposes using a low-precision quantization of random Fourier features (LP-RFFs) to build a high-rank approximation under a memory budget, and shows quantization has a negligible effect on generalization performance in important settings.
• Computer Science
IEEE Transactions on Pattern Analysis and Machine Intelligence
• 2022
This survey systematically review the work on random features from the past ten years and discusses the relationship between random features and modern over-parameterized deep neural networks (DNNs), including the use of high dimensional random features in the analysis of DNNs as well as the gaps between current theoretical and empirical results.
• Computer Science, Mathematics
J. Mach. Learn. Res.
• 2016
A new discrepancy measure called box discrepancy is derived based on theoretical characterizations of the integration error with respect to a given sequence based on explicit box discrepancy minimization in Quasi-Monte Carlo (QMC) approximations.
• Computer Science
NIPS
• 2014
This work proposes the first fast oblivious subspace embeddings that are able to embed a space induced by a non-linear kernel without explicitly mapping the data to the high-dimensional space.