Fisher discriminant analysis with kernels
@article{Mika1999FisherDA,
title={Fisher discriminant analysis with kernels},
author={Sebastian Mika and Gunnar R{\"a}tsch and Jason Weston and Bernhard Scholkopf and K.R. Mullers},
journal={Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468)},
year={1999},
pages={41-48}
}A non-linear classification technique based on Fisher's discriminant is proposed. The main ingredient is the kernel trick which allows the efficient computation of Fisher discriminant in feature space. The linear classification in feature space corresponds to a (powerful) non-linear decision function in input space. Large scale simulations demonstrate the competitiveness of our approach.
2,912 Citations
Kernel Fisher Discriminant Analysis Embedded with Feature Selection
- Computer Science2007 International Conference on Machine Learning and Cybernetics
- 2007
Experimental results clearly show that the new kernel Fisher discriminant analysis embedded with feature selection can greatly reduce the dimensions of the inputs, without harm to the classification results.
Invariant Feature Extraction and Classification in Kernel Spaces
- Computer Science, MathematicsNIPS
- 1999
Employing a unified framework in terms of a nonlinear variant of the Rayleigh coefficient, this work proposes non-linear generalizations of Fisher's discriminant and oriented PCA using Support Vector kernel functions.
Two Variations on Fisher's Linear Discriminant for Pattern Recognition
- Computer ScienceIEEE Trans. Pattern Anal. Mach. Intell.
- 2002
This paper provides two fast and simple techniques for improving on the classification performance provided by Fisher's linear discriminant for two classes by extended to nonlinear decision surfaces through the use of Mercer kernels.
Discriminant kernels based support vector machine
- Computer ScienceThe First Asian Conference on Pattern Recognition
- 2011
KDA is one of the nonlinear extensions of Linear Discriminant Analysis and the kernel function is usually defined a priori and it is not known what the optimum kernel function for nonlinear discriminant analysis is.
Non-parametric Fisher's discriminant analysis with kernels for data classification
- Computer SciencePattern Recognit. Lett.
- 2013
A sequential approach for multi-class discriminant analysis with kernels
- Computer Science2004 IEEE International Conference on Acoustics, Speech, and Signal Processing
- 2004
This work presents a sequential algorithm for GDA avoiding problems when one deals with large numbers of datapoints when dealing with nonlinear discriminant analysis using kernel functions.
Kernel Discriminant Analysis Based Face Recognition
- Computer Science
- 2014
The kernel trick is used to represent the complicated nonlinear relationships of input data to develop kernel discriminant analysis (KDA) algorithm, a traditional dimensionality reduction technique for feature extraction.
An improved training algorithm for kernel Fisher discriminants
- Computer ScienceAISTATS
- 2001
We present a fast training algorithm for the kernel Fisher discriminant classifier. It uses a greedy approximation technique and has an empirical scaling behavior which improves upon the state of the…
Nonlinear Discriminant Analysis Using Kernel Functions
- Computer ScienceNIPS
- 1999
The presented algorithm allows a simple formulation of the EM-algorithm in terms of kernel functions which leads to a unique concept for unsupervised mixture analysis, supervised discriminant analysis and semi-supervised discriminantAnalysis with partially unlabelled observations in feature spaces.
Weighted maximum margin discriminant analysis with kernels
- Computer Science, EngineeringNeurocomputing
- 2005
References
SHOWING 1-10 OF 21 REFERENCES
Discriminant Analysis by Gaussian Mixtures
- Computer Science
- 1996
This paper fits Gaussian mixtures to each class to facilitate effective classification in non-normal settings, especially when the classes are clustered.
Discriminant Analysis and Statistical Pattern Recognition
- Business
- 1992
Provides a systematic account of the subject area, concentrating on the most recent advances in the field. While the focus is on practical considerations, both theoretical and practical issues are…
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
- MathematicsNeural Computation
- 1998
A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
Regularized Discriminant Analysis
- Computer Science
- 1989
Alternatives to the usual maximum likelihood estimates for the covariance matrices are proposed, characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk.
Flexible Discriminant Analysis by Optimal Scoring
- Mathematics, Computer Science
- 1994
Nonparametric versions of discriminant analysis are obtained by replacing linear regression by any nonparametric regression method so that any multiresponse regression technique can be postprocessed to improve its classification performance.
Simplified Support Vector Decision Rules
- Computer ScienceICML
- 1996
The results show that the method can decrease the computational complexity of the decision rule by a factor of ten with no loss in generalization perfor mance making the SVM test speed com petitive with that of other methods.
Bayes Optimal Hyperplanes ! Maximal Margin Hyperplanes
- Computer Science, Mathematics
- 1999
This work provides an alternative justification for maximal margin hyperplane classifiers by relating them to Bayes optimal classifiers that use Parzen windows estimations with Gaussian kernels, and explores the idea of using the hyperplane that is optimal relative to a density with some small non-zero kernel width, and presents some promising preliminary results.
Input space versus feature space in kernel-based methods
- Computer ScienceIEEE Trans. Neural Networks
- 1999
The geometry of feature space is reviewed, and the connection between feature space and input space is discussed by dealing with the question of how one can, given some vector in feature space, find a preimage in input space.
Support vector learning
- Computer Science
- 1997
This book provides a comprehensive analysis of what can be done using Support vector Machines, achieving record results in real-life pattern recognition problems, and proposes a new form of nonlinear Principal Component Analysis using Support Vector kernel techniques, which it is considered as the most natural and elegant way for generalization of classical Principal Component analysis.
Introduction to Statistical Pattern Recognition
- Psychology
- 1972
This completely revised second edition presents an introduction to statistical pattern recognition, which is appropriate as a text for introductory courses in pattern recognition and as a reference book for workers in the field.