# Kernel Design Using Boosting

@inproceedings{Crammer2002KernelDU, title={Kernel Design Using Boosting}, author={K. Crammer and Joseph Keshet and Y. Singer}, booktitle={NIPS}, year={2002} }

The focus of the paper is the problem of learning kernel operators from empirical data. We cast the kernel design problem as the construction of an accurate kernel from simple (and less accurate) base kernels. We use the boosting paradigm to perform the kernel construction process. To do so, we modify the booster so as to accommodate kernel operators. We also devise an efficient weak-learner for simple kernels that is based on generalized eigen vector decomposition. We demonstrate the… Expand

#### 142 Citations

Learning with Idealized Kernels

- Mathematics, Computer Science
- ICML
- 2003

This paper considers the problem of adapting the kernel so that it becomes more similar to the so-called ideal kernel as a distance metric learning problem that searches for a suitable linear transform (feature weighting) in the kernel-induced feature space. Expand

Linear kernel combination using boosting

- Mathematics, Computer Science
- ESANN
- 2012

This paper proposes a novel algorithm to design multi- class kernels based on an iterative combination of weak kernels in a schema inspired from the boosting framework and evaluates its method for classification on a toy example and comparison with a reference iterative kernel design method. Expand

Semi-supervised mixture of kernels via LPBoost methods

- Mathematics, Computer Science
- Fifth IEEE International Conference on Data Mining (ICDM'05)
- 2005

By modifying the column generation boosting algorithm LPBoost to a more general linear programming formulation, this work is able to efficiently solve mixture-of-kernel problems and automatically select kernel basis functions centered at labeled data as well as unlabeled data. Expand

Column-generation boosting methods for mixture of kernels

- Computer Science, Mathematics
- KDD
- 2004

A boosting approach to classification and regression based on column generation using a mixture of kernels, which produces sparser solutions, and thus significantly reduces the testing time and is able to scale CG boosting to large datasets. Expand

A survey of the state of the art in learning the kernels

- Computer Science
- Knowledge and Information Systems
- 2011

An overview of algorithms to learn the kernel is presented and a comparison of various approaches to find an optimal kernel is provided to help identify pivotal issues that lead to efficient design of such algorithms. Expand

Kernels: Regularization and Optimization

- Computer Science
- 2005

It is shown that for several machine learning tasks, such as binary classification, regression and novelty detection, the resulting optimization problem is a semidefinite program, and theoretical and experimental evidence is provided to support the idea of regularization by early stopping of conjugate gradient type algorithms. Expand

Learning the kernel matrix by maximizing a KFD-based class separability criterion

- Mathematics, Computer Science
- Pattern Recognit.
- 2007

This paper proposes a novel method for learning the kernel matrix based on maximizing a class separability criterion that is similar to those used by linear discriminant analysis (LDA) and kernel Fisher discriminant (KFD). Expand

Learning the Kernel with Hyperkernels

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2005

The equivalent representer theorem for the choice of kernels is state and a semidefinite programming formulation of the resulting optimization problem is presented, which leads to a statistical estimation problem similar to the problem of minimizing a regularized risk functional. Expand

Improving efficiency of multi-kernel learning for support vector machines

- Computer Science
- 2008 International Conference on Machine Learning and Cybernetics
- 2008

This study reformulate the SDP problem to reduce the time and space requirements, and strategies for reducing the search space in solving the SSPD problem are introduced. Expand

Low-Dimensional Feature Learning with Kernel Construction

- 2011

We propose a practical method of semi-supervised feature learning with constructed kernels from combinations of non-linear weak rankers for classification applications. While in kernel methods one… Expand

#### References

SHOWING 1-10 OF 21 REFERENCES

On the Complexity of Learning the Kernel Matrix

- Computer Science, Mathematics
- NIPS
- 2002

A suitable way of constraining the class is proposed and an efficient algorithm is used to solve the resulting optimization problem, preliminary experimental results are presented, and an alignment-based approach is compared. Expand

Input space versus feature space in kernel-based methods

- Mathematics, Medicine
- IEEE Trans. Neural Networks
- 1999

The geometry of feature space is reviewed, and the connection between feature space and input space is discussed by dealing with the question of how one can, given some vector in feature space, find a preimage in input space. Expand

On Kernel-Target Alignment

- Computer Science
- NIPS
- 2001

The notion of kernel-alignment, a measure of similarity between two kernel functions or between a kernel and a target function, is introduced, giving experimental results showing that adapting the kernel to improve alignment on the labelled data significantly increases the alignment on a test set, giving improved classification accuracy. Expand

Learning the Kernel Matrix with Semidefinite Programming

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2004

This paper shows how the kernel matrix can be learned from data via semidefinite programming (SDP) techniques and leads directly to a convex method for learning the 2-norm soft margin parameter in support vector machines, solving an important open problem. Expand

Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By

- 2000

The main and important contribution of this paper is in establishing a connection between boosting, a newcomer to the statistics scene, and additive models. One of the main properties of boosting… Expand

The Spectrum Kernel: A String Kernel for SVM Protein Classification

- Mathematics, Computer Science
- Pacific Symposium on Biocomputing
- 2002

A new sequence-similarity kernel, the spectrum kernel, is introduced for use with support vector machines (SVMs) in a discriminative approach to the protein classification problem and performs well in comparison with state-of-the-art methods for homology detection. Expand

An Introduction to Support Vector Machines and Other Kernel-based Learning Methods

- Computer Science, Mathematics
- 2000

This is the first comprehensive introduction to Support Vector Machines (SVMs), a new generation learning system based on recent advances in statistical learning theory, and will guide practitioners to updated literature, new applications, and on-line software. Expand

Text Classification using String Kernels

- Computer Science
- J. Mach. Learn. Res.
- 2002

A novel kernel is introduced for comparing two text documents consisting of an inner product in the feature space consisting of all subsequences of length k, which can be efficiently evaluated by a dynamic programming technique. Expand

Logistic Regression, AdaBoost and Bregman Distances

- Mathematics, Computer Science
- Machine Learning
- 2004

A unified account of boosting and logistic regression in which each learning problem is cast in terms of optimization of Bregman distances, and a parameterized family of algorithms that includes both a sequential- and a parallel-update algorithm as special cases are described, thus showing how the sequential and parallel approaches can themselves be unified. Expand

An introduction to Support Vector Machines

- Computer Science
- 2000

This book is the first comprehensive introduction to Support Vector Machines (SVMs), a new generation learning system based on recent advances in statistical learning theory. The book also introduces… Expand