Multicategory Support Vector Machines

  title={Multicategory Support Vector Machines},
  author={Yoonkyung Lee and Yi Lin and Grace Wahba},
  journal={Journal of the American Statistical Association},
  pages={67 - 81}
Two-category support vector machines (SVM) have been very popular in the machine learning community for classification problems. Solving multicategory problems by a series of binary classifiers is quite common in the SVM paradigm; however, this approach may fail under various circumstances. We propose the multicategory support vector machine (MSVM), which extends the binary SVM to the multicategory case and has good theoretical properties. The proposed method provides a unifying framework when… 

Cloud Classification of Satellite Radiance Data by Multicategory Support Vector Machines

The proposed MSVM in addition provides a unifying framework when there are either equal or unequal misclassification costs, and when there is a possibly nonrepresentative training set.

Ensemble Approaches of Support Vector Machines for Multiclass Classification

Two novel ensemble approaches are presented: probabilistic ordering of one-vs-rest (OVR) SVMs with naive Bayes classifier and multiple decision templates of OVR SVMs.

A New Multiclass Support Vector Machine

A new multiclass Support Vector Machine (SVM) is presented, which can be used to find the optimal decision boundaries in a multiclass classification problem, and it is believed that the proposed method provides a promising new way of looking at multiclass Classification problems.

The consistency of multicategory support vector machines

The goal of classification is to construct a classifier with small misclassification error, and the consistency of MSVMs is established under a mild condition, where the universal consistency holds true if the reproducing kernel Hilbert space is dense in C norm.

Support vector machines: A distance-based approach to multi-class classification

  • Wissam AoudiA. Barbar
  • Computer Science
    2016 IEEE International Multidisciplinary Conference on Engineering Technology (IMCET)
  • 2016
This paper proposes a new approach to constructing the multi-class classification function, where the structure and properties of the support vectors are exploited without altering the training procedure, and argues that the proposed distance- based method represents a more rigorous and intuitive measure than the traditional hyperplane-based method.

Support Vector Machines for Unbalanced Multicategory Classification

This work proposes a robust method to treat unbalanced cases based on the weights of the class and shows that the proposed method is effective to analyze unbalanced proportion data.

Reinforced Angle-Based Multicategory Support Vector Machines

  • Chong ZhangYufeng LiuJunhui WangHongtu Zhu
  • Computer Science
    Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America
  • 2016
It is proved that RAMSVMs can enjoy Fisher consistency, and it is shown that the RAMSVM can be implemented using the very efficient coordinate descent algorithm on its dual problem, using an angle-based prediction rule with k − 1 functions directly.

Multiclass Approaches for Support Vector Machine Based Land Cover Classification

  • M. Pal
  • Computer Science, Environmental Science
  • 2008
Results from this study conclude the usefulness of One vs. One multi class approach in term of accuracy and computational cost over other multi class approaches.

Support Vector Machine Implementations for Classification & Clustering

Benefits of theinternal-SVM approach, along with further refinements to the internal-multiclass SVM algorithms that offer significant improvement in training time without sacrificing accuracy are described.

Fuzzy Pairwise Multiclass Support Vector Machines

This paper proposes a new fuzzy model that includes the advantages of several previously published methods solving their drawbacks and yields membership degrees in the unit interval and in some cases, improves the performance of the former methods in the unclassified regions.



Cloud Classification of Satellite Radiance Data by Multicategory Support Vector Machines

The proposed MSVM in addition provides a unifying framework when there are either equal or unequal misclassification costs, and when there is a possibly nonrepresentative training set.

Multicategory Classification by Support Vector Machines

This work investigates how two-class discrimination methods can be extended to the multiclass case, and shows how the linear programming (LP) and quadratic programming (QP) approaches based on Vapnik's Support Vector Machine (SVM) can be combined to yield two new approaches to theMulticlass problem.

Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers

A general method for combining the classifiers generated on the binary problems is proposed, and a general empirical multiclass loss bound is proved given the empirical loss of the individual binary learning algorithms.

Support vector machines for multi-class pattern recognition

A formulation of the SVM is proposed that enables a multi-class pattern recognition problem to be solved in a single optimisation and a similar generalization of linear programming machines is proposed.

Introduction to Support Vector Machines

Support Vector Machines (SVM’s) are intuitive, theoretically wellfounded, and have shown to be practically successful.

Statistical Properties and Adaptive Tuning of Support Vector Machines

An approach to adaptively tuning the smoothing parameter(s) in the SVMs is described, based on the generalized approximate cross validation (GACV), which is an easily computable proxy of the GCKL.

From Margins to Probabilities in Multiclass Learning Problems

A new decoding function is proposed that combines the margins through an estimate of their class conditional probabilities, which leads to good estimates of kernel parameters and the corresponding classifiers attain high accuracy.

SSVM: A Smooth Support Vector Machine for Classification

Smoothing methods are applied here to generate and solve an unconstrained smooth reformulation of the support vector machine for pattern classification using a completely arbitrary kernel, which converges globally and quadratically.


The extended algorithm provides a computational shortcut to attain the entire spectrum of solutions from the most regularized to the completely overtted ones.

Optimal Properties and Adaptive Tuning of Standard and Nonstandard Support Vector Machines

We review some of the basic ideas of Support Vector Machines (SVM’s) for classification, with the goal of describing how these ideas can sit comfortably inside the statistical literature in decision