Why averaging classifiers can protect against overfitting

@inproceedings{Freund2001WhyAC,
  title={Why averaging classifiers can protect against overfitting},
  author={Yoav Freund and Yishay Mansour and Robert E. Schapire},
  booktitle={AISTATS},
  year={2001}
}
We study a simple learning algorithm for binary classification. Instead of predicting with the best hypothesis in the hypothesis class, this algorithm predicts w th a weighted average of all hypotheses, weighted exponentially with respect to their training error. We show that the prediction of this algorithm is much more stable than the predicti on of an algorithm that predicts with the best hypothesis. By allowing the algorithm to abst ain from predicting on some examples, we show that the… CONTINUE READING
Highly Cited
This paper has 26 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.

Explore Further: Topics Discussed in This Paper

References

Publications referenced by this paper.
Showing 1-10 of 20 references

McAllester . Some PACBayesian theorems

A. David
Proceedings of the Eleventh Annual Conference on Computational Learning Theory • 1998

Friedman . On bias , variance , 0 / 1loss , and thecurseofdimensionality

H. Jerome
Data Mining and Knowledge Discovery • 1997

On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality

Data Mining and Knowledge Discovery • 1997
View 1 Excerpt

Warmuth . How to use expert advice

Nicolò Cesa-Bianchi, Yoav Freund, +3 authors K. Manfred
Journal of the Association for Computing Machinery • 1997

Williamson . A pac analysis of a bayesian estimator

John Shawe-Taylor, C. Robert
Proceedings of the Tenth Annual Conference on Computational Learning Theory • 1997

A Probabilistic Theory of Pattern Recognition

Luc Devroye, Ĺazló Györfi, Ǵabor Lugosi
1996

Bagging predictors

Machine Learning • 1996
View 2 Excerpts

Similar Papers

Loading similar papers…