The evidence framework applied to support vector machines

@article{Kwok2000TheEF,
  title={The evidence framework applied to support vector machines},
  author={James Tin-Yau Kwok},
  journal={IEEE transactions on neural networks},
  year={2000},
  volume={11 5},
  pages={
          1162-73
        }
}
  • J. Kwok
  • Published 1 September 2000
  • Computer Science
  • IEEE transactions on neural networks
In this paper, we show that training of the support vector machine (SVM) can be interpreted as performing the level 1 inference of MacKay's evidence framework.We further on show that levels 2 and 3 of the evidence framework can also be applied to SVMs. This integration allows automatic adjustment of the regularization parameter and the kernel parameter to their near-optimal values. Moreover, it opens up a wealth of Bayesian tools for use with SVMs. Performance of this method is evaluated on… 
View on PubMed
cse.ust.hk
155 Citations
Highly Influential Citations
11
Background Citations
33
Methods Citations
52
Results Citations
2

155 Citations

Citation Type

Citation Type

Applying the Bayesian Evidence Framework to \nu -Support Vector Regression
TLDR
It is shown that ν-SVR training implies a prior on the size of the Ɛ-tube that is dependent on the number of training patterns, and this prior has properties that are in line with the error-regulating behavior of ν.
Moderating the outputs of support vector machine classifiers
  • J. Kwok
  • Computer Science
    IEEE Trans. Neural Networks
  • 1999
TLDR
This paper extends the use of moderated outputs to the support vector machine (SVM) by making use of a relationship between SVM and the evidence framework, and the moderated output derived here can be taken as an approximation to the posterior class probability.
On Kernel Selection in Relevance Vector Machines Using Stability Principle
In this paper we propose an alternative interpretation of Bayesian learning based on maximal evidence principle. We establish a notion of local evidence which can be viewed as a compromise between…
Bayesian Support Vector Regression
TLDR
It is shown that the Bayesian evidence framework can be applied to both -support vector regression ( -SVR) and -support vectors regression algorithms, without the need of a validation set.
A new Bayesian design method for support vector classification
  • Wei Chu, S. Keerthi, C. Ong
  • Computer Science
    Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02.
  • 2002
TLDR
A novel differentiable loss function called trigonometric loss function with the desirable characteristic of natural normalization in the likelihood function is proposed, and standard Gaussian processes techniques are followed to set up a Bayesian framework.
Analysis of detectors for support vector machines and least square support vector machines
  • A. Kuh
  • Computer Science
    Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290)
  • 2002
TLDR
The performance capabilities of the support vector machine and the least squares SVM for a two hypothesis detection problem is discussed and a Bayesian framework where there are priors associated with each hypothesis and costs for making decisions is considered.
Extended Bayesian framework for automatic tuning of kernel data-mining methods
TLDR
This paper proposes a local evidence estimation method which establishes a compromise between accuracy and stability of algorithm, and presents model of kernel algorithms as well as approximations for evidence estimation, and gives results of experimental evaluation.
Model selection for support vector machine classification
Bayesian Inference in Trigonometric Support Vector Classifier
TLDR
A novel differentiable loss function called trigonometric loss function with the desirable characteristics of natural normalization in the likelihood function is proposed, and a Bayesian framework in stationary Gaussian stochastic processes is described.
Fast Bayesian support vector machine parameter tuning with the Nystrom method
  • C. Gold, Peter Sollich
  • Computer Science
    Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005.
  • 2005
TLDR
The Nystrom approximation to the SVM kernel significantly reduces the dimensionality of the space to be simulated in the HMC, and gives excellent performance in comparison with other approaches to hyperparameter tuning.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 24 REFERENCES
Moderating the outputs of support vector machine classifiers
  • J. Kwok
  • Computer Science
    IEEE Trans. Neural Networks
  • 1999
TLDR
This paper extends the use of moderated outputs to the support vector machine (SVM) by making use of a relationship between SVM and the evidence framework, and the moderated output derived here can be taken as an approximation to the posterior class probability.
The Evidence Framework Applied to Classification Networks
  • D. Mackay
  • Computer Science
    Neural Computation
  • 1992
TLDR
It is demonstrated that the Bayesian framework for model comparison described for regression models in MacKay (1992a,b) can also be applied to classification problems and an information-based data selection criterion is derived and demonstrated within this framework.
Bayesian Interpolation
  • D. Mackay
  • Computer Science
    Neural Computation
  • 1992
TLDR
The Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data by examining the posterior probability distribution of regularizing constants and noise levels.
Predicting Time Series with Support Vector Machines
TLDR
Two different cost functions for Support Vectors are made use: training with an e insensitive loss and Huber's robust loss function and how to choose the regularization parameters in these models are discussed.
Bayesian Learning for Neural Networks
TLDR
Bayesian Learning for Neural Networks shows that Bayesian methods allow complex neural network models to be used without fear of the "overfitting" that can occur with traditional neural network learning methods.
Support vector learning
TLDR
This book provides a comprehensive analysis of what can be done using Support vector Machines, achieving record results in real-life pattern recognition problems, and proposes a new form of nonlinear Principal Component Analysis using Support Vector kernel techniques, which it is considered as the most natural and elegant way for generalization of classical Principal Component analysis.
Information-Based Objective Functions for Active Data Selection
TLDR
Within a Bayesian learning framework, objective functions are discussed that measure the expected informativeness of candidate measurements that depend on the assumption that the hypothesis space is correct.
Bayesian Regularization in Constructive Neural Networks
TLDR
Simulation shows that regularization, with input training using a full Bayesian approach, produces networks with better generalization performance and lower susceptibility to over-fitting as the network size increases.
Computing with Infinite Networks
For neural networks with a wide class of weight-priors, it can be shown that in the limit of an infinite number of hidden units the prior over functions tends to a Gaussian process. In this paper…
Computation with Infinite Neural Networks
For neural networks with a wide class of weight priors, it can be shown that in the limit of an infinite number of hidden units, the prior over functions tends to a gaussian process. In this article,…
...
1
2
3
...