Soft nearest prototype classification

@article{Seo2003SoftNP,
  title={Soft nearest prototype classification},
  author={Sambu Seo and Mathias Bode and Klaus Obermayer},
  journal={IEEE transactions on neural networks},
  year={2003},
  volume={14 2},
  pages={
          390-8
        }
}
We propose a new method for the construction of nearest prototype classifiers which is based on a Gaussian mixture ansatz and which can be interpreted as an annealed version of learning vector quantization (LVQ). The algorithm performs a gradient descent on a cost-function minimizing the classification error on the training set. We investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task. Results show 1) that… 
View on PubMed
doi.org
97 Citations
Highly Influential Citations
12
Background Citations
40
Methods Citations
27

97 Citations

Citation Type

Citation Type

Metric Learning for Prototype-Based Classification
In this chapter, one of themost popular and intuitive prototype-based classification algorithms, learning vector quantization (LVQ), is revisited, and recent extensions towards automatic metric…
Hyperparameter learning in probabilistic prototype-based models
Regularized margin-based conditional log-likelihood loss for prototype learning
Nearest prototype classification of noisy data
TLDR
This work shows that the ENPC algorithm is able to obtain solutions with high classification success even when data is noisy, and a comparison with optimal (hand made) solutions and other different classification algorithms demonstrates the good performance of theENPC algorithm in accuracy and number of prototypes as the noise level increases.
University of Groningen Advanced methods for prototype-based classification
TLDR
A regularization technique to extend recently proposed matrix learning schemes in Learning Vector Quantization (LVQ) by extending the cost function by an appropriate regularization term prevents the unfavorable behavior and can help to improve the generalization ability.
Prototype-based Models for the Supervised Learning of Classification Schemes
TLDR
An introduction to the use of prototype-based models in supervised machine learning is given, to represent previously observed data in terms of so-called prototypes, which reflect typical properties of the data.
Matrix Learning in Learning Vector Quantization
We propose a new matrix learning scheme to extend Generalized Relevance Learning Vector Quantization (GRLVQ), an efficient prototype-based classification algorithm. By introducing a full matrix of…
Distance Measures for Prototype Based Classification
The basic concepts of distance based classification are introduced in terms of clear-cut example systems. The classical k-Nearest-Neigbhor (kNN) classifier serves as the starting point of the…
Adaptive Relevance Matrices in Learning Vector Quantization
We propose a new matrix learning scheme to extend relevance learning vector quantization (RLVQ), an efficient prototype-based classification algorithm, toward a general adaptive metric. By…
Dynamical analysis of LVQ type learning rules
TLDR
Using concepts from statistical physics and the theory of on-line learning, this work presents a rigorous mathematical investigation of the dynamics of LVQ type classifiers in a prototypical scenario and can observe significant differences of the algorithmic stability and generalization ability.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 29 REFERENCES
Improved versions of learning vector quantization
  • T. Kohonen
  • Computer Science
    1990 IJCNN International Joint Conference on Neural Networks
  • 1990
TLDR
It has turned out that the differences between the presented algorithms in regard to the remaining discretization error are not significant, and thus the choice of the algorithm may be based on secondary arguments, such as stability in learning, in which respect the variant introduced (LVQ2.1) seems to be superior to the others.
Prototype-based minimum classification error/generalized probabilistic descent training for various speech units
TLDR
This work extends LVQ into a prototype-based minimum error classifier appropriate for the classification of various speech units which the original LVQ was unable to treat, and discusses the issue of smoothing the loss function from the perspective of increasing classifier robustness.
New discriminative training algorithms based on the generalized probabilistic descent method
TLDR
A family of new discriminative training algorithms can be rigorously formulated for various kinds of classifier frameworks, including the popular dynamic time warping (DTW) and hidden Markov model (HMM).
LVQ_PAK: The Learning Vector Quantization Program Package
TLDR
The LVQ PAK program package contains all programs necessary for the correct application of certain Learning Vector Quantization algorithms in an arbitrary statistical classiication or pattern recognition task, as well as a program for the monitoring of the codebook vectors at any time during the learning process.
Discriminative learning for minimum error classification [pattern recognition]
TLDR
A fundamental technique for designing a classifier that approaches the objective of minimum classification error in a more direct manner than traditional methods is given and is contrasted with several traditional classifier designs in typical experiments to demonstrate the superiority of the new learning formulation.
A global optimization technique for statistical classifier design
TLDR
The theoretical basis for the annealing method is derived, on which it is based the development of a novel design algorithm and its effectiveness and superior performance in the design of practical classifiers for some of the most popular structures currently in use are demonstrated.
Classifying with Gaussian Mixtures and Clusters
TLDR
The proposed constrained rank Gaussian mixtures model and the WTA approximations improve the performance over the unconstrained models and clustering based algorithms like LVQ and k-Means are derived.
Application of a generalized probabilistic descent method to dynamic time warping-based speech recognition
  • Takashi Komori, S. Katagiri
  • Computer Science
    [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing
  • 1992
TLDR
A generalized probabilistic descent method (GPD) is evaluated in designing a speech recognizer incorporated with the dynamic time warping methodology and results clearly demonstrate that GPD can be a viable candidate for a method to realize a high-performance speech recognizers.
Gaussian process regression: active data selection and test point rejection
  • Sambu Seo, Marko Wallat, T. Graepel, K. Obermayer
  • Mathematics
    Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium
  • 2000
TLDR
It is found that, for both a two-dimensional toy problem and a real-world benchmark problem, the variance is a reasonable criterion for both active data selection and test point rejection.
Information-Based Objective Functions for Active Data Selection
TLDR
Within a Bayesian learning framework, objective functions are discussed that measure the expected informativeness of candidate measurements that depend on the assumption that the hypothesis space is correct.
...
1
2
3
...