Trends & Controversies: Support Vector Machines

@article{Hearst1998TrendsC,
  title={Trends \& Controversies: Support Vector Machines},
  author={Marti A. Hearst},
  journal={IEEE Intell. Syst.},
  year={1998},
  volume={13},
  pages={18-28}
}
My first exposure to Support Vector Machines came this spring when heard Sue Dumais present impressive results on text categorization using this analysis technique. This issue's collection of essays should help familiarize our readers with this interesting new racehorse in the Machine Learning stable. Bernhard Scholkopf, in an introductory overview, points out that a particular advantage of SVMs over other learning algorithms is that it can be analyzed theoretically using concepts from… 
View via Publisher
svms.org
2,786 Citations
Highly Influential Citations
125
Background Citations
187
Methods Citations
384
Results Citations
2

Figures and Tables from this paper

2,786 Citations

Using support vector machines for automatic new topic identification

Findings of this study suggest that support vector machines' performance depends on the characteristics of the dataset it is applied on.

Performance Analysis of Naiotave Bayes Classification, Support Vector Machines and Neural Networks for Spam Categorization

The performances of Support Vector Machines, Neural Networks, and Naive Bayes techniques are compared using a publicly available corpus (LINGSPAM) for different cost scenarios and the results show that NN has significantly better performance than the two other, having acceptable training times.

Using wavelet analysis for text categorization in digital libraries: a first experiment with Strathprints

A pilot experiment of replacing standard test collections by a set of 6,000 objects from a real-world digital repository, indexed by Library of Congress Subject Headings, and test support vector machines in a supervised learning setting for their ability to reproduce the existing classification suggests that wavelet-based kernels slightly outperformed traditional kernels on classification reconstruction from abstracts and vice versa from full-text documents.

Fringe SVM Settings and Aggressive Feature Reduction

Statistical techniques for aggressive feature reduction are studied on data obtained in agene knock-out experiment and it is shown that relatively superior performance of fringe SVMs can be observed on regular textmining data bases, such as Reuters newswire benchmark, if only the less frequent features are used.

ML-SPD: Machine Learning based Sentiment Polarity Detection

Different text representation models are explored in conjunction with state-of-the-art traditional Machine Learning techniques: Support Vector Machines (SVM), Neural Networks (NN), Nave Bayes (NB), and combination of NB and SVM classifier (NBSVM) to solve SPD problem.

On Text-based Mining with Active Learning and Background Knowledge Using SVM

Experimental results show that the proposed techniques present a considerable improvement in classification performance, even when small labeled training sets are available.

English & Persian Document Categorization based on a Novel Hybrid Algorithm

Experimental evaluation of automatic document categorization based on large text hierarchy by taking into account hierarchical structure of examples and using feature selection for large text data shows that this approach gives promising results and can potentially be used for document classification on the Web.

E-cient Pattern Selection for Support Vector Classiflers and its CRM Application

This thesis proposes neighborhood property based pattern selection algorithm (NPPS) which selects the patterns near the decision boundary based on the neighborhood properties which is implemented as a naive form with time complexity O(M) where M is the number of given training patterns.

Do We Need More Training Samples For Text Classification?

In this article, feature selection as a technology to remove irrelevant features was found be able to prevent overfitting to a great extent and the kappa coefficient as a performance measure of classifiers could increase 11 points at the maximum.

Provably Fast Training Algorithms for Support Vector Machines

An upper bound on the expected running time is formally proved which is quasilinear with respect to the number of data points and polynomial withrespect to the other parameters, i.e., the number and inverse of a chosen soft margin parameter.
...

86 References

Transductive Inference for Text Classification using Support Vector Machines

An analysis of why Transductive Support Vector Machines are well suited for text classi cation is presented, and an algorithm for training TSVMs, handling 10,000 examples and more is proposed.

Support Vector Machines: Training and Applications

Preliminary results are presented obtained applying SVM to the problem of detecting frontal human faces in real images, and the main idea behind the decomposition is the iterative solution of sub-problems and the evaluation of, and also establish the stopping criteria for the algorithm.

Structural Risk Minimization Over Data-Dependent Hierarchies

A result is presented that allows one to trade off errors on the training sample against improved generalization performance, and a more general result in terms of "luckiness" functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets.

Advances in kernel methods: support vector learning

Support vector machines for dynamic reconstruction of a chaotic system, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel.

Training support vector machines: an application to face detection

  • E. OsunaR. FreundF. Girosi
  • Computer Science
    Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • 1997
A decomposition algorithm that guarantees global optimality, and can be used to train SVM's over very large data sets is presented, and the feasibility of the approach on a face detection problem that involves a data set of 50,000 data points is demonstrated.

Text Categorization with Support Vector Machines: Learning with Many Relevant Features

This paper explores the use of Support Vector Machines (SVMs) for learning text classifiers from examples. It analyzes the particular properties of learning with text data and identifies why SVMs are…

Simplified Support Vector Decision Rules

The results show that the method can decrease the computational complexity of the decision rule by a factor of ten with no loss in generalization perfor mance making the SVM test speed com petitive with that of other methods.

Extracting Support Data for a Given Task

It is observed that three different types of handwritten digit classifiers construct their decision surface from strongly overlapping small subsets of the data base, which opens up the possibility of compressing data bases significantly by disposing of theData which is not important for the solution of a given task.

Properties of Support Vector Machines

It is shown that the decision surface can be written as the sum of two orthogonal terms, the first depending on only the margin vectors (which are SVs lying on the margin), the second proportional to the regularization parameter for almost all values of the parameter.

An improved training algorithm for support vector machines

  • E. OsunaR. FreundF. Girosi
  • Computer Science
    Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop
  • 1997
This paper presents a decomposition algorithm that is guaranteed to solve the QP problem and that does not make assumptions on the expected number of support vectors.
...