• Corpus ID: 10778991

Relative Error Embeddings of the Gaussian Kernel Distance

@inproceedings{Chen2017RelativeEE,
  title={Relative Error Embeddings of the Gaussian Kernel Distance},
  author={Di Chen and J. M. Phillips},
  booktitle={ALT},
  year={2017}
}
A reproducing kernel can define an embedding of a data point into an infinite dimensional reproducing kernel Hilbert space (RKHS). The norm in this space describes a distance, which we call the kernel distance. The random Fourier features (of Rahimi and Recht) describe an oblivious approximate mapping into finite dimensional Euclidean space that behaves similar to the RKHS. We show in this paper that for the Gaussian kernel the Euclidean norm between these mapped to features has $(1+\epsilon… 

Figures from this paper

Relative Error RKHS Embeddings for Gaussian Kernels
TLDR
The main insight is to effectively modify the well-traveled random Fourier features to be slightly biased and have higher variance, but so they can be defined as a convolution over the function space.
The GaussianSketch for Almost Relative Error Kernel Distance
We introduce two versions of a new sketch for approximately embedding the Gaussian kernel into Euclidean inner product space. These work by truncating infinite expansions of the Gaussian kernel, and
N ov 2 01 8 Relative Error RKHS Embeddings for Gaussian Kernels
TLDR
The main insight is to effectively modify the well-traveled random Fourier features to be slightly biased and have higher variance, but so they can be defined as a convolution over the function space.
Fourier Sparse Leverage Scores and Approximate Kernel Learning
TLDR
New explicit upper bounds on the leverage scores of Fourier sparse functions under both the Gaussian and Laplace measures are proved, which generalize existing work that only applies to uniformly distributed data.
Provably Useful Kernel Matrix Approximation in Linear Time
We give the first algorithms for kernel matrix approximation that run in time linear in the number of data points and output an approximation which gives provable guarantees when used in many
Sketched MinDist
TLDR
This paper shows how large this set of sketch vectors needs to be under a variety of shapes and scenarios and provides direct connection to the sensitivity sample framework, so relative error can be preserved in $d$ dimensions using Q = O(d/\varepsilon^2)$.
A probability distribution kernel based on whitening transformation
TLDR
Experiments show that the DPWT kernel exhibits superior performances compared to the other state of the art kernels, and can effectively eliminate the correlation between vectors, and reduce the redundancy of data which can further improve the accuracy of classification.
Coresets for Kernel Clustering
TLDR
The main result is a coreset for kernel k-MEANS that works for a general kernel and has size poly(k −1).
Real-Valued Embeddings and Sketches for Fast Distance and Similarity Estimation
TLDR
Methods and algorithms for fast estimation of data distance/similarity measures from formed real-valued vectors of small dimension using random projection and sampling are considered.

References

SHOWING 1-10 OF 40 REFERENCES
Hilbert Space Embeddings and Metrics on Probability Measures
TLDR
It is shown that the distance between distributions under γk results from an interplay between the properties of the kernel and the distributions, by demonstrating that distributions are close in the embedding space when their differences occur at higher frequencies.
Comparing distributions and shapes using the kernel distance
TLDR
This paper presents fast approximation algorithms for computing the kernel distance between two point sets P and Q that runs in near-linear time in the size of P ∪ Q (an explicit calculation would take quadratic time).
Subspace Embeddings for the Polynomial Kernel
TLDR
This work proposes the first fast oblivious subspace embeddings that are able to embed a space induced by a non-linear kernel without explicitly mapping the data to the high-dimensional space.
Sketching, Embedding, and Dimensionality Reduction for Information Spaces
TLDR
This paper shows that information distances can be sketched efficiently in the aggregate streaming model, and proves a dimensionality reduction result for the Hellinger, Jensen--Shannon, and $\chi^2$ divergences that preserves the information geometry of the distributions (specifically, by retaining the simplex structure of the space).
Random Fourier Approximations for Skewed Multiplicative Histogram Kernels
TLDR
The concept of translation-invariance and the random Fourier feature methodology is extended to arbitrary, locally compact Abelian groups and a new group called the skewed-multiplicative group is proposed and designed to design translation- Invariant kernels on it.
Optimal Rates for Random Fourier Features
TLDR
A detailed finite-sample theoretical analysis about the approximation quality of RFFs is provided by establishing optimal (in terms of the RFF dimension, and growing set size) performance guarantees in uniform norm, and presenting guarantees in Lr (1 ≤ r < ∞) norms.
On the Error of Random Fourier Features
TLDR
The uniform error bound of that paper on random Fourier features is improved, as well as giving novel understandings of the embedding's variance, approximation error, and use in some machine learning methods.
Compact Random Feature Maps
TLDR
The error bounds of CRAFT maps are proved demonstrating their superior kernel reconstruction performance compared to the previous approximation schemes, and it is shown how structured random matrices can be used to efficiently generate CRAFTMaps.
Geometric Inference on Kernel Density Estimates
TLDR
It is shown that geometric inference of a point cloud can be calculated by examining its kernel density estimate with a Gaussian kernel, and an algorithm to estimate its topology using weighted Vietoris-Rips complexes is provided.
Kernel k-means: spectral clustering and normalized cuts
TLDR
The generality of the weighted kernel k-means objective function is shown, and the spectral clustering objective of normalized cut is derived as a special case, leading to a novel weightedkernel k-Means algorithm that monotonically decreases the normalized cut.
...
...