Kernels for Vector-Valued Functions: a Review

  title={Kernels for Vector-Valued Functions: a Review},
  author={Mauricio A {\'A}lvarez and Lorenzo Rosasco and Neil D. Lawrence},
Kernel methods are among the most popular techniques in machine learning. From a regularization perspective they play a central role in regularization theory as they provide a natural choice for the hypotheses space and the regularization functional through the notion of reproducing kernel Hilbert spaces. From a probabilistic perspective they are the key in the context of Gaussian processes, where the kernel function is known as the covariance function. Traditionally, kernel methods have been… 

Figures from this paper

Online Learning with Multiple Operator-valued Kernels

Two online algorithms for learning a vector-valued function f while taking into account the output structure are described, one of which extends the standard kernel-based online learning algorithm NORMA from scalar-valued to operator-valued setting and the other addresses the limitation of pre-defining theoutput structure in ONORMA by learning sequentially a linear combination of operator- valued kernels.

Random Fourier Features For Operator-Valued Kernels

A general principle for Operator-valued Random Fourier Feature construction relies on a generalization of Bochner's theorem for translation-invariant operator-valued Mercer kernels and proves the uniform convergence of the kernel approximation for bounded and unbounded operator random Fourier features.

Multi-task Learning in Vector-valued Reproducing Kernel Banach Spaces with the $\ell^1$ Norm.

A class of vector-valued reproducing kernel Banach spaces with the $\ell^1$ norm is constructed so that the constructed spaces could have desirable properties including the crucial linear representer theorem.

Kernel Mean Embedding of Distributions: A Review and Beyonds

A comprehensive review of existing work and recent advances in the Hilbert space embedding of distributions, and to discuss the most challenging issues and open problems that could lead to new research directions.

Operator-valued Kernels for Learning from Functional Response Data

In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label is

Learning with Operator-valued Kernels in Reproducing Kernel Krein Spaces

This work considers operator-valued kernels which might not be necessarily positive definite, and an iterative Operator based Minimum Residual (OpMINRES) algorithm is proposed for solving the loss stabilization problem.

Online Learning with Operator-valued Kernels

An online algorithm, OLOK, is described that extends the standard kernel-based online learning algorithm NORMA from scalar-valued to operator-valued setting and reports a cumulative error bound that holds both for classification and regression.

On the Dualization of Operator-Valued Kernel Machines

This work investigates how to use the duality principle to handle different families of loss functions, yet unexplored within vv-RKHSs.

Nonlinear Functional Output Regression: a Dictionary Approach

In many fields, each data instance consists in a high number of measurements of the same underlying phenomenon. Such high dimensional data generally enjoys strong smoothness across features which can



Kernels for Multi--task Learning

This paper provides a foundation for multi-task learning using reproducing kernel Hilbert spaces of vector-valued functions using classes of matrix- valued kernels which are linear and are of the dot product or the translation invariant type.

Universal Multi-Task Kernels

The primary goal here is to derive conditions which ensure that the kernel K is universal, which means that on every compact subset of the input space, every continuous function with values in Y can be uniformly approximated by sections of the kernel.

On Learning Vector-Valued Functions

This letter provides a study of learning in a Hilbert space of vector-valued functions and derives the form of the minimal norm interpolant to a finite set of data and applies it to study some regularization functionals that are important in learning theory.

Characterizing the Function Space for Bayesian Kernel Models

A coherent Bayesian kernel model based on an integral operator defined as the convolution of a kernel with a signed measure is studied, creating a function theoretic foundation for using non-parametric prior specifications in Bayesian modeling, such as Gaussian process and Dirichlet process prior distributions.

Learning Multiple Tasks with Kernel Methods

The experiments show that learning multiple related tasks simultaneously using the proposed approach can significantly outperform standard single-task learning particularly when there are many related tasks but few data per task.

Multi-output learning via spectral filtering

A finite sample bound for the excess risk of the obtained estimator allows to prove consistency both for regression and multi-category classification and some promising results of the proposed algorithms are presented.

Some Properties of Regularized Kernel Methods

A quantitative version of the representer theorem holding for both regression and classification, for both differentiable and non-differentiable loss functions, and for arbitrary offset terms is proved.

Gaussian Processes for Machine Learning

The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics, and deals with the supervised learning problem for both regression and classification.

Convex multi-task feature learning

It is proved that the method for learning sparse representations shared across multiple tasks is equivalent to solving a convex optimization problem for which there is an iterative algorithm which converges to an optimal solution.

Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

  • A. Atiya
  • Computer Science
    IEEE Transactions on Neural Networks
  • 2005
This book is an excellent choice for readers who wish to familiarize themselves with computational intelligence techniques or for an overview/introductory course in the field of computational intelligence.