Kernel methods in machine learning

@article{Hofmann2008KernelMI,
  title={Kernel methods in machine learning},
  author={Thomas Hofmann and Bernhard Scholkopf and Alex Smola},
  journal={Annals of Statistics},
  year={2008},
  volume={36},
  pages={1171-1220}
}
We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on… Expand

Figures from this paper

On the use of kernel functions in Minimal Learning Machines
The Minimal Learning Machine is a recently proposed supervised method in which learning consists of fitting a multiresponse linear regression model between distances computed from the input andExpand
Learning in Reproducing Kernel Hilbert Spaces
This chapter is dedicated to nonparametric modeling of nonlinear functions in reproducing kernel Hilbert spaces (RKHS). The basic definitions and concepts behind RKH spaces are presented, includingExpand
New empirical nonparametric kernels for support vector machine classification
TLDR
A general procedure is suggested to produce nonparametric and efficient kernels by finding an empirical and theoretical connection between positive semidefinite matrices and certain metric space properties. Expand
Kernel Methods for Structured Data
TLDR
Kernel methods are a class of non-parametric learning techniques relying on kernels that allow to decouple the representation of the data from the specific learning algorithm, provided it can be defined in terms of distance or similarity between instances. Expand
A stable hyperparameter selection for the Gaussian RBF kernel for discrimination
Kernel-based classification methods, for example, support vector machines, map the data into a higher-dimensional space via a kernel function. In practice, choosing the value of hyperparameter in theExpand
Locally Adaptive Kernel Estimation Using Sparse Functional Programming
TLDR
This work proposes to locally adapt the RKHS (more specifically, its smoothness parameter) over which it seeks to perform function estimation by using a sparse functional program and must solve an infinite dimensional, non-convex optimization problem. Expand
Feature vector regression with efficient hyperparameters tuning and geometric interpretation
TLDR
The main contribution of this paper is the new kernel method, capable of achieving satisfactory results with reduced efforts because of the small number of hyperparameters to be tuned and the reduced training dataset size used. Expand
Learning Rates for l1-Regularized Kernel Classifiers
We consider a family of classification algorithms generated from a regularization kernel scheme associated with -regularizer and convex loss function. Our main purpose is to provide an explicitExpand
Asymmetric Kernel Learning
This paper addresses a new kernel learning problem, referred to as ‘asymmetric kernel learning’ (AKL). First, we give the definition of asymmetric kernel and point out that many ‘similarityExpand
A survey of the state of the art in learning the kernels
TLDR
An overview of algorithms to learn the kernel is presented and a comparison of various approaches to find an optimal kernel is provided to help identify pivotal issues that lead to efficient design of such algorithms. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 176 REFERENCES
A survey of kernels for structured data
TLDR
This survey describes several approaches of defining positive definite kernels on structured instances directly on the basis of areal vector space and thus in a single table. Expand
A Review of Kernel Methods in Machine Learning
TLDR
The present review aims to summarize the state of the art on a conceptual level for positive definite kernels by presenting various approaches for estimating dependencies and analyzing data that make use of kernels and the use of reproducing kernel Hilbert spaces as a means to define statistical models. Expand
Learning Kernel Classifiers - Theory and Algorithms
  • R. Herbrich
  • Computer Science
  • Adaptive computation and machine learning
  • 2002
TLDR
This book provides the first comprehensive overview of both the theory and algorithms of kernel classifiers, including the most recent developments, and a detailed introduction to learning theory, including VC and PAC-Bayesian theory, data-dependent structural risk minimization, and compression bounds. Expand
On the Influence of the Kernel on the Consistency of Support Vector Machines
TLDR
It is shown that the soft margin algorithms with universal kernels are consistent for a large class of classification problems including some kind of noisy tasks provided that the regularization parameter is chosen well. Expand
Support Vector Machines are Universally Consistent
TLDR
It is shown that the 1-norm soft margin classifier with Gaussian RBF kernel on a compact subset X of Rd and regularization parameter cn = nβ-1 is universally consistent, if n is the training set size and 0 >β> 1/d. Expand
Max-Margin Markov Networks
TLDR
Maximum margin Markov (M3) networks incorporate both kernels, which efficiently deal with high-dimensional features, and the ability to capture correlations in structured data, and a new theoretical bound for generalization in structured domains is provided. Expand
Convolution kernels on discrete structures
We introduce a new method of constructing kernels on sets whose elements are discrete structures like strings, trees and graphs. The method can be applied iteratively to build a kernel on a innniteExpand
Kernel independent component analysis
  • F. Bach, Michael I. Jordan
  • Computer Science, Mathematics
  • 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).
  • 2003
TLDR
A class of algorithms for independent component analysis which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space is presented, showing that these algorithms outperform many of the presently known algorithms. Expand
A kernel method for multi-labelled classification
TLDR
This article presents a Support Vector Machine like learning system to handle multi-label problems, based on a large margin ranking system that shares a lot of common properties with SVMs. Expand
Kernel Dependency Estimation
TLDR
This work considers the learning problem of finding a dependency between a general class of objects and another, possibly different, generalclass of objects, made possible by employing similarity measures in both input and output spaces using kernel functions, thus embedding the objects into vector spaces. Expand
...
1
2
3
4
5
...