# Support Vector Machines and the Bayes Rule in Classification

@article{Lin2004SupportVM, title={Support Vector Machines and the Bayes Rule in Classification}, author={Yi Lin}, journal={Data Mining and Knowledge Discovery}, year={2004}, volume={6}, pages={259-275} }

The Bayes rule is the optimal classification rule if the underlying distribution of the data is known. In practice we do not know the underlying distribution, and need to “learn” classification rules from the data. One way to derive classification rules in practice is to implement the Bayes rule approximately by estimating an appropriate classification function. Traditional statistical methods use estimated log odds ratio as the classification function. Support vector machines (SVMs) are one…

## 268 Citations

### Statistical Properties and Adaptive Tuning of Support Vector Machines

- Computer ScienceMachine Learning
- 2004

An approach to adaptively tuning the smoothing parameter(s) in the SVMs is described, based on the generalized approximate cross validation (GACV), which is an easily computable proxy of the GCKL.

### A Note on Margin-based Loss Functions in Classification by

- Computer Science
- 2002

It is shown that the hinge loss is the tightest convex upper bound of the misclassification loss, and the Fisher consistency of margin-based loss functions often leads to consistency and rate of convergence (to the Bayes optimal risk) results under general conditions.

### A Note on Margin-based Loss Functions in Classification by Yi Lin A Note on Margin-based Loss Functions in Classification

- Computer Science
- 2007

It is shown that the hinge loss is the tightest convex upper bound of the misclassification loss, and the Fisher consistency of margin-based loss functions often leads to consistency and rate of convergence (to the Bayes optimal risk) results under general conditions.

### Statistical performance of support vector machines

- Computer Science
- 2008

The main result shows that it is possible to obtain fast rates of convergence for SVMs and builds on the observation made by other authors that the SVM can be viewed as a statistical regularization procedure.

### Automatically computed document dependent weighting factor facility for Naïve Bayes classification

- Computer ScienceExpert Syst. Appl.
- 2010

### Comparing naive Bayes, decision trees, and SVM with AUC and accuracy

- Computer ScienceThird IEEE International Conference on Data Mining
- 2003

It is proved that AUC is, in general, a better measure (defined precisely) than accuracy for evaluating performance of learning algorithms.

### NORM SUPPORT VECTOR MACHINE

- Computer Science
- 2008

It is shown that the F∞-norm SVM is equivalent to a linear programming problem and can be efficiently solved using standard techniques and enjoys competitive performance when compared with the 1-norm and 2-norm SVMs.

### Large‐margin classification with multiple decision rules

- Computer ScienceStat. Anal. Data Min.
- 2016

This paper proposes a spectrum of statistical learning problems that span the hard and soft classification tasks based on fitting multiple decision rules to the data, and reveals a novel collection of learning tasks of increasing complexity.

### Support Vector Machines with Applications

- Computer Science
- 2006

This paper is intended as an introduction to SVMs and their applications, emphasizing their key features, and some algorithmic extensions and illustrative real-world applications of SVMs are shown.

### Multiclass Probability Estimation With Support Vector Machines

- Computer ScienceJournal of Computational and Graphical Statistics
- 2019

This article proposes a simple yet effective framework to endow kernel SVMs with the feature of multiclass probability estimation and demonstrates competitive performance of the new estimator when compared with existing methods such as multiple logistic regression, linear discrimination analysis, tree-based methods, and random forest, under various classification settings.

## References

SHOWING 1-10 OF 19 REFERENCES

### Support Vector Machines for Classification in Nonstandard Situations

- Computer ScienceMachine Learning
- 2004

This paper explains why the standard support vectors machine is not suitable for the nonstandard situation, and introduces a simple procedure for adapting the support vector machine methodology to the non standard situation.

### On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality

- MathematicsData Mining and Knowledge Discovery
- 2004

This work candramatically mitigate the effect of the bias associated with some simpleestimators like “naive” Bayes, and the bias induced by the curse-of-dimensionality on nearest-neighbor procedures.

### Robust Bounds on Generalization from the Margin Distribution

- Computer Science, Mathematics
- 1998

It is shown that a slight generalization of their construction can be used to give a pac style bound on the tail of the distribution of the generalization errors that arise from a given sample size.

### Support-Vector Networks

- Computer ScienceMachine Learning
- 2004

High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

### Tensor product space ANOVA models in multivariate function estimation

- Mathematics, Computer Science
- 1998

The main result is that, in a variety of general nonparametric problems, under general conditions, the rate of convergence for the penalized likelihood estimator in the TPS-ANOVA model is O([n(logn)1 r] 2m 2m+1 ) when the smoothing parameter is appropriately chosen, which means that curse of dimensionality is overcome by the tensor product space ANOVA models.

### Tensor product space ANOVA models

- Mathematics, Computer Science
- 2000

The quick optimal rate of the TPS-ANOVA model makes it very preferable in high-dimensional function estimation, and many properties of the tensor product space of Sobolev-Hilbert spaces are given.

### Asymptotic Analysis of Penalized Likelihood and Related Estimators

- Mathematics
- 1990

A general approach to the first order asymptotic analysis ofpenalized likelihood and related estimators is described. The method gives expansions for the systematic and random error. Asymptotic…

### A Unified Framework for Regularization Networks and Support Vector Machines

- Computer Science
- 1999

This work presents regularization Networks and Support Vector Machines in a unified framework in the context of Vapnik''s theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics.

### What is a support vector machine?

- BiologyNature Biotechnology
- 2006

Support vector machines (SVMs) are becoming popular in a wide variety of biological applications. But, what exactly are SVMs and how do they work? And what are their most promising applications in…

### The Nature of Statistical Learning Theory

- Computer ScienceStatistics for Engineering and Information Science
- 2000

Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing…