Statistical Properties and Adaptive Tuning of Support Vector Machines

@article{Lin2002StatisticalPA,
  title={Statistical Properties and Adaptive Tuning of Support Vector Machines},
  author={Yi Lin and Grace Wahba and Hao Zhang and Yoonkyung Lee},
  journal={Machine Learning},
  year={2002},
  volume={48},
  pages={115-136}
}
In this paper we consider the statistical aspects of support vector machines (SVMs) in the classification context, and describe an approach to adaptively tuning the smoothing parameter(s) in the SVMs. The relation between the Bayes rule of classification and the SVMs is discussed, shedding light on why the SVMs work well. This relation also reveals that the misclassification rate of the SVMs is closely related to the generalized comparative Kullback-Leibler distance (GCKL) proposed in Wahba… 
View on Springer
link.springer.com
50 Citations
Highly Influential Citations
1
Background Citations
15
Methods Citations
15

50 Citations

Optimal Properties and Adaptive Tuning of Standard and Nonstandard Support Vector Machines

We review some of the basic ideas of Support Vector Machines (SVM’s) for classification, with the goal of describing how these ideas can sit comfortably inside the statistical literature in decision…

Multicategory Support Vector Machines

The MSVM is proposed, which extends the binary SVM to the multicategory case and has good theoretical properties, and an approximate leave-one-out cross-validation function is derived, analogous to the binary case.

Coherence functions with applications in large-margin classification methods

A family of coherence functions, which are convex and differentiable, as surrogates of the hinge function are proposed and studied, which refer to the use of the coherence function in large-margin classification as "C-learning," and efficient coordinate descent algorithms for the training of regularized C-learning models are presented.

Variable selection for support vector machines via smoothing spline anova

This work proposes a new type of regularization to conduct simultaneous clas- sication and variable selection in the SVM, using the sum of functional component norms, which automatically applies soft-thresholding operations to functional components, hence yields sparse solutions.

Variable selection for SVM via smoothing spline ANOVA

This work proposes a new type of regularization to conduct simultaneous classification and variable selection in the SVM, under the framework of smoothing spline ANOVA models, which automatically applies soft-thresholding operations to functional components hence yields sparse solutions.

Chapter 4 – Nonlinear Classifiers

Chaotic antlion algorithm for parameter optimization of support vector machine

The experimental results proved that the proposed Chaotic Antlion Optimization (CALO-SVM) algorithm is capable of finding the optimal values of the SVM parameters and avoids the local optima problem.

Support Vector Machine Classification for High Dimensional Microarray Data Analysis, With Applications in Cancer Research

This chapter first reviews the basic principles of the SVM and its variant formulations, then demonstrates how these methods overcome the curse of dimensionality and thus, become suitable to accurately identify differentially expressed gene signatures and build reliable classification models in cancer research.

Statistical Learning in Medical Data Analysis

This article provides a tour of statistical learning regularization methods that have found application in a variety of medical data analysis problems and involves an optimization problem which balances fidelity to the data with complexity of the model.

Model building with likelihood basis pursuit

It is shown how slice-modeling techniques significantly improve the efficiency of individual solves and thus speed-up the grid search, and how derivative-free optimization algorithms can find better solutions with fewer function evaluations by seeding them with a coarse grid search.

References

SHOWING 1-10 OF 21 REFERENCES

Support Vector Machines and the Bayes Rule in Classification

  • Yi Lin
  • Computer Science
    Data Mining and Knowledge Discovery
  • 2004
It is shown that the asymptotic target of SVMs are some interesting classification functions that are directly related to the Bayes rule, and helps understand the success of SVM in many classification studies, and makes it easier to compare SVMs and traditional statistical methods.

Advances in kernel methods: support vector learning

Support vector machines for dynamic reconstruction of a chaotic system, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel.

A training algorithm for optimal margin classifiers

A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions,…

Support-Vector Networks

High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

Knowledge-based analysis of microarray gene expression data by using support vector machines.

A method of functionally classifying genes by using gene expression data from DNA microarray hybridization experiments, based on the theory of support vector machines (SVMs), to predict functional roles for uncharacterized yeast ORFs based on their expression data is introduced.

Asymptotic Analysis of Penalized Likelihood and Related Estimators

A general approach to the first order asymptotic analysis ofpenalized likelihood and related estimators is described. The method gives expansions for the systematic and random error. Asymptotic…

The Nature of Statistical Learning Theory

  • V. Vapnik
  • Computer Science
    Statistics for Engineering and Information Science
  • 2000
Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing…

A Sparse Representation for Function Approximation

We derive a new general representation for a function as a linear combination of local correlation kernels at optimal sparse locations (and scales) and characterize its relation to principal…

Construction and Assessment of Classification Rules

We may not be able to make you love reading, but construction and assessment of classification rules will lead you to love reading starting from now. Book is the window to open the new world. The…

A Tutorial on Support Vector Machines for Pattern Recognition

The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, w...