# Kernel methods in machine learning

@article{Hofmann2008KernelMI, title={Kernel methods in machine learning}, author={Thomas Hofmann and Bernhard Scholkopf and Alex Smola}, journal={Annals of Statistics}, year={2008}, volume={36}, pages={1171-1220} }

We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on… Expand

#### Figures from this paper

#### 1,006 Citations

On the use of kernel functions in Minimal Learning Machines

- Chemistry
- 2018

The Minimal Learning Machine is a recently proposed supervised method in which learning consists of fitting a multiresponse linear regression model between distances computed from the input and… Expand

Learning in Reproducing Kernel Hilbert Spaces

- Mathematics
- 2015

This chapter is dedicated to nonparametric modeling of nonlinear functions in reproducing kernel Hilbert spaces (RKHS). The basic definitions and concepts behind RKH spaces are presented, including… Expand

New empirical nonparametric kernels for support vector machine classification

- Mathematics, Computer Science
- Appl. Soft Comput.
- 2013

A general procedure is suggested to produce nonparametric and efficient kernels by finding an empirical and theoretical connection between positive semidefinite matrices and certain metric space properties. Expand

Kernel Methods for Structured Data

- Computer Science
- Handbook on Neural Information Processing
- 2013

Kernel methods are a class of non-parametric learning techniques relying on kernels that allow to decouple the representation of the data from the specific learning algorithm, provided it can be defined in terms of distance or similarity between instances. Expand

A stable hyperparameter selection for the Gaussian RBF kernel for discrimination

- Mathematics
- 2010

Kernel-based classification methods, for example, support vector machines, map the data into a higher-dimensional space via a kernel function. In practice, choosing the value of hyperparameter in the… Expand

Locally Adaptive Kernel Estimation Using Sparse Functional Programming

- Computer Science
- 2018 52nd Asilomar Conference on Signals, Systems, and Computers
- 2018

This work proposes to locally adapt the RKHS (more specifically, its smoothness parameter) over which it seeks to perform function estimation by using a sparse functional program and must solve an infinite dimensional, non-convex optimization problem. Expand

Feature vector regression with efficient hyperparameters tuning and geometric interpretation

- Mathematics, Computer Science
- Neurocomputing
- 2016

The main contribution of this paper is the new kernel method, capable of achieving satisfactory results with reduced efforts because of the small number of hyperparameters to be tuned and the reduced training dataset size used. Expand

Learning Rates for l1-Regularized Kernel Classifiers

- Mathematics, Computer Science
- J. Appl. Math.
- 2013

We consider a family of classification algorithms generated from a regularization kernel scheme associated with -regularizer and convex loss function. Our main purpose is to provide an explicit… Expand

Asymmetric Kernel Learning

- Mathematics
- 2010

This paper addresses a new kernel learning problem, referred to as ‘asymmetric kernel learning’ (AKL). First, we give the definition of asymmetric kernel and point out that many ‘similarity… Expand

A survey of the state of the art in learning the kernels

- Computer Science
- Knowledge and Information Systems
- 2011

An overview of algorithms to learn the kernel is presented and a comparison of various approaches to find an optimal kernel is provided to help identify pivotal issues that lead to efficient design of such algorithms. Expand

#### References

SHOWING 1-10 OF 176 REFERENCES

A survey of kernels for structured data

- Computer Science
- SKDD
- 2003

This survey describes several approaches of defining positive definite kernels on structured instances directly on the basis of areal vector space and thus in a single table. Expand

A Review of Kernel Methods in Machine Learning

- Computer Science
- 2006

The present review aims to summarize the state of the art on a conceptual level for positive definite kernels by presenting various approaches for estimating dependencies and analyzing data that make use of kernels and the use of reproducing kernel Hilbert spaces as a means to define statistical models. Expand

Learning Kernel Classifiers - Theory and Algorithms

- Computer Science
- Adaptive computation and machine learning
- 2002

This book provides the first comprehensive overview of both the theory and algorithms of kernel classifiers, including the most recent developments, and a detailed introduction to learning theory, including VC and PAC-Bayesian theory, data-dependent structural risk minimization, and compression bounds. Expand

On the Influence of the Kernel on the Consistency of Support Vector Machines

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2001

It is shown that the soft margin algorithms with universal kernels are consistent for a large class of classification problems including some kind of noisy tasks provided that the regularization parameter is chosen well. Expand

Support Vector Machines are Universally Consistent

- Computer Science, Mathematics
- J. Complex.
- 2002

It is shown that the 1-norm soft margin classifier with Gaussian RBF kernel on a compact subset X of Rd and regularization parameter cn = nβ-1 is universally consistent, if n is the training set size and 0 >β> 1/d. Expand

Max-Margin Markov Networks

- Computer Science
- NIPS
- 2003

Maximum margin Markov (M3) networks incorporate both kernels, which efficiently deal with high-dimensional features, and the ability to capture correlations in structured data, and a new theoretical bound for generalization in structured domains is provided. Expand

Convolution kernels on discrete structures

- Computer Science
- 1999

We introduce a new method of constructing kernels on sets whose elements are discrete structures like strings, trees and graphs. The method can be applied iteratively to build a kernel on a innnite… Expand

Kernel independent component analysis

- Computer Science, Mathematics
- 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).
- 2003

A class of algorithms for independent component analysis which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space is presented, showing that these algorithms outperform many of the presently known algorithms. Expand

A kernel method for multi-labelled classification

- Computer Science
- NIPS
- 2001

This article presents a Support Vector Machine like learning system to handle multi-label problems, based on a large margin ranking system that shares a lot of common properties with SVMs. Expand

Kernel Dependency Estimation

- Computer Science, Mathematics
- NIPS
- 2002

This work considers the learning problem of finding a dependency between a general class of objects and another, possibly different, generalclass of objects, made possible by employing similarity measures in both input and output spaces using kernel functions, thus embedding the objects into vector spaces. Expand