The Perceptron Algorithm vs. Winnow: Linear vs. Logarithmic Mistake Bounds when few Input Variables are Relevant

@inproceedings{Kivinen1995ThePA,
  title={The Perceptron Algorithm vs. Winnow: Linear vs. Logarithmic Mistake Bounds when few Input Variables are Relevant},
  author={Jyrki Kivinen and Manfred K. Warmuth},
  booktitle={COLT},
  year={1995}
}
We give an adversary strategy that forces the Perceptron algorithm to make (N-k+1)/2 mistakes when learning k-literal disjunctions over N variables. Experimentally we see that even for simple random data, the number of mistakes made by the Perceptron algorithm grows almost linearly with N, even if the number k of relevant variable remains a small constant. In contrast, Littlestone''s algorithm Winnow makes at most O(k log N) mistakes for the same problem. Both algorithms use thresholded linear… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 79 CITATIONS

Some Formal Analysis of Rocchio's Similarity-Based Relevance Feedback Algorithm

  • Information Retrieval
  • 2002
VIEW 16 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Computational Learning Theory

  • Lecture Notes in Computer Science
  • 2002
VIEW 4 EXCERPTS
CITES RESULTS, BACKGROUND & METHODS

Learning to Recognize Three-Dimensional Objects

  • Neural Computation
  • 2002
VIEW 4 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Tracking the Best Linear Predictor

  • J. Mach. Learn. Res.
  • 2001
VIEW 3 EXCERPTS
CITES BACKGROUND & METHODS

A New Approximate Maximal Margin Classification Algorithm

VIEW 3 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Neural Networks and Deep Learning

  • Springer International Publishing
  • 2018
VIEW 1 EXCERPT
CITES METHODS

Enhancing the performance of SVM based document classifier by selecting good class representative using fuzzy membership criteria

  • 2017 3rd International Conference on Computational Intelligence & Communication Technology (CICT)
  • 2017

FILTER CITATIONS BY YEAR

1996
2018

CITATION STATISTICS

  • 4 Highly Influenced Citations