On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines


In this paper we describe the algorithmic implementation of multiclass kernel-based vector machines. Our starting point is a generalized notion of the margin to multiclass problems. Using this notion we cast multiclass categorization problems as a constrained optimization problem with a quadratic objective function. Unlike most of previous approaches which typically decompose a multiclass problem into multiple independent binary classification tasks, our notion of margin yields a direct method for training multiclass predictors. By using the dual of the optimization problem we are able to incorporate kernels with a compact set of constraints and decompose the dual problem into multiple optimization problems of reduced size. We describe an efficient fixed-point algorithm for solving the reduced optimization problems and prove its convergence. We then discuss technical details that yield significant running time improvements for large datasets. Finally, we describe various experiments with our approach comparing it to previously studied kernel-based methods. Our experiments indicate that for multiclass problems we attain state-of-the-art accuracy.

Extracted Key Phrases

8 Figures and Tables

Showing 1-10 of 32 references

Making large-scale support vector machine learning practical Advances in Kernel Methods -Support Vector Learning

  • Thorsten Joachims
  • 1998
Highly Influential
5 Excerpts
Showing 1-10 of 1,002 extracted citations
Citations per Year

2,355 Citations

Semantic Scholar estimates that this publication has received between 2,073 and 2,670 citations based on the available data.

See our FAQ for additional information.