Fisher discriminant analysis with kernels

@article{Mika1999FisherDA,
  title={Fisher discriminant analysis with kernels},
  author={Sebastian Mika and Gunnar R{\"a}tsch and Jason Weston and Bernhard Scholkopf and K.R. Mullers},
  journal={Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468)},
  year={1999},
  pages={41-48}
}
  • S. Mika, Gunnar Rätsch, K.R. Mullers
  • Published 23 August 1999
  • Computer Science
  • Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468)
A non-linear classification technique based on Fisher's discriminant is proposed. The main ingredient is the kernel trick which allows the efficient computation of Fisher discriminant in feature space. The linear classification in feature space corresponds to a (powerful) non-linear decision function in input space. Large scale simulations demonstrate the competitiveness of our approach. 

Figures and Tables from this paper

Kernel Fisher Discriminant Analysis Embedded with Feature Selection
  • Yong-Qiao Wang
  • Computer Science
    2007 International Conference on Machine Learning and Cybernetics
  • 2007
TLDR
Experimental results clearly show that the new kernel Fisher discriminant analysis embedded with feature selection can greatly reduce the dimensions of the inputs, without harm to the classification results.
Invariant Feature Extraction and Classification in Kernel Spaces
TLDR
Employing a unified framework in terms of a nonlinear variant of the Rayleigh coefficient, this work proposes non-linear generalizations of Fisher's discriminant and oriented PCA using Support Vector kernel functions.
Two Variations on Fisher's Linear Discriminant for Pattern Recognition
  • T. Cooke
  • Computer Science
    IEEE Trans. Pattern Anal. Mach. Intell.
  • 2002
TLDR
This paper provides two fast and simple techniques for improving on the classification performance provided by Fisher's linear discriminant for two classes by extended to nonlinear decision surfaces through the use of Mercer kernels.
Discriminant kernels based support vector machine
TLDR
KDA is one of the nonlinear extensions of Linear Discriminant Analysis and the kernel function is usually defined a priori and it is not known what the optimum kernel function for nonlinear discriminant analysis is.
A sequential approach for multi-class discriminant analysis with kernels
TLDR
This work presents a sequential algorithm for GDA avoiding problems when one deals with large numbers of datapoints when dealing with nonlinear discriminant analysis using kernel functions.
Kernel Discriminant Analysis Based Face Recognition
TLDR
The kernel trick is used to represent the complicated nonlinear relationships of input data to develop kernel discriminant analysis (KDA) algorithm, a traditional dimensionality reduction technique for feature extraction.
An improved training algorithm for kernel Fisher discriminants
We present a fast training algorithm for the kernel Fisher discriminant classifier. It uses a greedy approximation technique and has an empirical scaling behavior which improves upon the state of the
Nonlinear Discriminant Analysis Using Kernel Functions
TLDR
The presented algorithm allows a simple formulation of the EM-algorithm in terms of kernel functions which leads to a unique concept for unsupervised mixture analysis, supervised discriminant analysis and semi-supervised discriminantAnalysis with partially unlabelled observations in feature spaces.
Weighted maximum margin discriminant analysis with kernels
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 21 REFERENCES
Discriminant Analysis by Gaussian Mixtures
TLDR
This paper fits Gaussian mixtures to each class to facilitate effective classification in non-normal settings, especially when the classes are clustered.
Discriminant Analysis and Statistical Pattern Recognition
Provides a systematic account of the subject area, concentrating on the most recent advances in the field. While the focus is on practical considerations, both theoretical and practical issues are
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
TLDR
A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
Regularized Discriminant Analysis
TLDR
Alternatives to the usual maximum likelihood estimates for the covariance matrices are proposed, characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk.
Flexible Discriminant Analysis by Optimal Scoring
TLDR
Nonparametric versions of discriminant analysis are obtained by replacing linear regression by any nonparametric regression method so that any multiresponse regression technique can be postprocessed to improve its classification performance.
Simplified Support Vector Decision Rules
TLDR
The results show that the method can decrease the computational complexity of the decision rule by a factor of ten with no loss in generalization perfor mance making the SVM test speed com petitive with that of other methods.
Bayes Optimal Hyperplanes ! Maximal Margin Hyperplanes
TLDR
This work provides an alternative justification for maximal margin hyperplane classifiers by relating them to Bayes optimal classifiers that use Parzen windows estimations with Gaussian kernels, and explores the idea of using the hyperplane that is optimal relative to a density with some small non-zero kernel width, and presents some promising preliminary results.
Input space versus feature space in kernel-based methods
TLDR
The geometry of feature space is reviewed, and the connection between feature space and input space is discussed by dealing with the question of how one can, given some vector in feature space, find a preimage in input space.
Support vector learning
TLDR
This book provides a comprehensive analysis of what can be done using Support vector Machines, achieving record results in real-life pattern recognition problems, and proposes a new form of nonlinear Principal Component Analysis using Support Vector kernel techniques, which it is considered as the most natural and elegant way for generalization of classical Principal Component analysis.
Introduction to Statistical Pattern Recognition
TLDR
This completely revised second edition presents an introduction to statistical pattern recognition, which is appropriate as a text for introductory courses in pattern recognition and as a reference book for workers in the field.
...
1
2
3
...