On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines

Abstract

In this paper we describe the algorithmic implementation of multiclass kernel-based vector machines. Our starting point is a generalized notion of the margin to multiclass problems. Using this notion we cast multiclass categorization problems as a constrained optimization problem with a quadratic objective function. Unlike most of previous approaches which typically decompose a multiclass problem into multiple independent binary classification tasks, our notion of margin yields a direct method for training multiclass predictors. By using the dual of the optimization problem we are able to incorporate kernels with a compact set of constraints and decompose the dual problem into multiple optimization problems of reduced size. We describe an efficient fixed-point algorithm for solving the reduced optimization problems and prove its convergence. We then discuss technical details that yield significant running time improvements for large datasets. Finally, we describe various experiments with our approach comparing it to previously studied kernel-based methods. Our experiments indicate that for multiclass problems we attain state-of-the-art accuracy.

Extracted Key Phrases

8 Figures and Tables

0100200'02'04'06'08'10'12'14'16
Citations per Year

2,361 Citations

Semantic Scholar estimates that this publication has 2,361 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@article{Crammer2001OnTA, title={On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines}, author={Koby Crammer and Yoram Singer}, journal={Journal of Machine Learning Research}, year={2001}, volume={2}, pages={265-292} }