# Kernels for Vector-Valued Functions: a Review

@article{lvarez2012KernelsFV,
title={Kernels for Vector-Valued Functions: a Review},
author={M. {\'A}lvarez and L. Rosasco and Neil Lawrence},
journal={Found. Trends Mach. Learn.},
year={2012},
volume={4},
pages={195-266}
}
• Published 2012
• Computer Science, Mathematics
• Found. Trends Mach. Learn.
Kernel methods are among the most popular techniques in machine learning. From a regularization perspective they play a central role in regularization theory as they provide a natural choice for the hypotheses space and the regularization functional through the notion of reproducing kernel Hilbert spaces. From a probabilistic perspective they are the key in the context of Gaussian processes, where the kernel function is known as the covariance function. Traditionally, kernel methods have been… Expand
478 Citations

#### Figures and Topics from this paper

Multi-task Learning in vector-valued reproducing kernel Banach spaces with the ℓ1 norm
• Computer Science, Mathematics
• J. Complex.
• 2021
This work constructs a class of vector-valued reproducing kernel Banach spaces with the l 1 norm to construct multi-task admissible kernels so that the constructed spaces could have desirable properties including the crucial linear representer theorem. Expand
Online Learning with Multiple Operator-valued Kernels
• Computer Science, Mathematics
• ArXiv
• 2013
Two online algorithms for learning a vector-valued function f while taking into account the output structure are described, one of which extends the standard kernel-based online learning algorithm NORMA from scalar-valued to operator-valued setting and the other addresses the limitation of pre-defining theoutput structure in ONORMA by learning sequentially a linear combination of operator- valued kernels. Expand
Random Fourier Features For Operator-Valued Kernels
• Mathematics, Computer Science
• ACML
• 2016
A general principle for Operator-valued Random Fourier Feature construction relies on a generalization of Bochner's theorem for translation-invariant operator-valued Mercer kernels and proves the uniform convergence of the kernel approximation for bounded and unbounded operator random Fourier features. Expand
Large-scale operator-valued kernel regression
This thesis proposes and study scalable methods to perform regression with Operator-Valued Kernels, and develops a general framework devoted to the approximation of shift-invariant MErcer kernels on Locally Compact Abelian groups. Expand
Multi-task Learning in Vector-valued Reproducing Kernel Banach Spaces with the $\ell^1$ Norm.
• Mathematics
• 2019
Targeting at sparse multi-task learning, we consider regularization models with an $\ell^1$ penalty on the coefficients of kernel functions. In order to provide a kernel method for this model, weExpand
Kernel Mean Embedding of Distributions: A Review and Beyonds
• Mathematics, Computer Science
• Found. Trends Mach. Learn.
• 2017
A comprehensive review of existing work and recent advances in the Hilbert space embedding of distributions, and to discuss the most challenging issues and open problems that could lead to new research directions. Expand
Operator-valued Kernels for Learning from Functional Response Data
• Computer Science, Mathematics
• J. Mach. Learn. Res.
• 2016
In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label isExpand
Learning with Operator-valued Kernels in Reproducing Kernel Krein Spaces
• Computer Science, Mathematics
• NeurIPS
• 2020
This work considers operator-valued kernels which might not be necessarily positive definite, and an iterative Operator based Minimum Residual (OpMINRES) algorithm is proposed for solving the loss stabilization problem. Expand
Online Learning with Operator-valued Kernels
• Computer Science
• ESANN
• 2015
An online algorithm, OLOK, is described that extends the standard kernel-based online learning algorithm NORMA from scalar-valued to operator-valued setting and reports a cumulative error bound that holds both for classification and regression. Expand
On the Dualization of Operator-Valued Kernel Machines
• Mathematics, Computer Science
• ArXiv
• 2019
This work investigates how to use the duality principle to handle different families of loss functions, yet unexplored within vv-RKHSs. Expand

#### References

SHOWING 1-10 OF 138 REFERENCES
• Computer Science
• NIPS
• 2004
This paper provides a foundation for multi-task learning using reproducing kernel Hilbert spaces of vector-valued functions using classes of matrix- valued kernels which are linear and are of the dot product or the translation invariant type. Expand
• Mathematics, Computer Science
• J. Mach. Learn. Res.
• 2008
The primary goal here is to derive conditions which ensure that the kernel K is universal, which means that on every compact subset of the input space, every continuous function with values in Y can be uniformly approximated by sections of the kernel. Expand
On Learning Vector-Valued Functions
• Mathematics, Computer Science
• Neural Computation
• 2005
This letter provides a study of learning in a Hilbert space of vector-valued functions and derives the form of the minimal norm interpolant to a finite set of data and applies it to study some regularization functionals that are important in learning theory. Expand
Characterizing the Function Space for Bayesian Kernel Models
• Mathematics, Computer Science
• J. Mach. Learn. Res.
• 2007
A coherent Bayesian kernel model based on an integral operator defined as the convolution of a kernel with a signed measure is studied, creating a function theoretic foundation for using non-parametric prior specifications in Bayesian modeling, such as Gaussian process and Dirichlet process prior distributions. Expand
Learning Multiple Tasks with Kernel Methods
• Mathematics, Computer Science
• J. Mach. Learn. Res.
• 2005
The experiments show that learning multiple related tasks simultaneously using the proposed approach can significantly outperform standard single-task learning particularly when there are many related tasks but few data per task. Expand
Multi-output learning via spectral filtering
• Mathematics, Computer Science
• Machine Learning
• 2012
A finite sample bound for the excess risk of the obtained estimator allows to prove consistency both for regression and multi-category classification and some promising results of the proposed algorithms are presented. Expand
Some Properties of Regularized Kernel Methods
• Mathematics, Computer Science
• J. Mach. Learn. Res.
• 2004
A quantitative version of the representer theorem holding for both regression and classification, for both differentiable and non-differentiable loss functions, and for arbitrary offset terms is proved. Expand
Kernel Methods for Pattern Analysis
• Computer Science
• 2004
The lectures will introduce the kernel methods approach to pattern analysis through the particular example of support vector machines for classification and argue that, ignoring the technical requirement of positive semi-definiteness, kernel design is not an unnatural task for a practitioner. Expand
Clustered Multi-Task Learning: A Convex Formulation
• Computer Science, Mathematics
• NIPS
• 2008
A new spectral norm is designed that encodes this a priori assumption that tasks are clustered into groups, which are unknown beforehand, and that tasks within a group have similar weight vectors, resulting in a new convex optimization formulation for multi-task learning. Expand
Gaussian Processes for Machine Learning
• Computer Science
• Adaptive computation and machine learning
• 2009
The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics, and deals with the supervised learning problem for both regression and classification. Expand