Training linear SVMs in linear time


Linear Support Vector Machines (SVMs) have become one of the most prominent machine learning techniques for high-dimensional sparse data commonly encountered in applications like text classification, word-sense disambiguation, and drug design. These applications involve a large number of examples <i>n</i> as well as a large number of features <i>N</i>, while each example has only <i>s</i> &lt;&lt; <i>N</i> non-zero features. This paper presents a Cutting Plane Algorithm for training linear SVMs that provably has training time <i>0(s,n)</i> for classification problems and <i>o</i>(<i>sn</i> log (<i>n</i>))for ordinal regression problems. The algorithm is based on an alternative, but equivalent formulation of the SVM optimization problem. Empirically, the Cutting-Plane Algorithm is several orders of magnitude faster than decomposition methods like svm light for large datasets.

DOI: 10.1145/1150402.1150429

Extracted Key Phrases

10 Figures and Tables

Citations per Year

2,100 Citations

Semantic Scholar estimates that this publication has 2,100 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Joachims2006TrainingLS, title={Training linear SVMs in linear time}, author={Thorsten Joachims}, booktitle={KDD}, year={2006} }