Additive Logistic Regression a Statistical View of Boosting

Abstract

Boosting Freund Schapire Schapire Singer is one of the most important recent developments in classi cation method ology The performance of many classi cation algorithms often can be dramatically improved by sequentially applying them to reweighted versions of the input data and taking a weighted majority vote of the sequence of classi ers thereby produced We show that this seemingly mysterious phenomenon can be understood in terms of well known statistical principles namely additive modeling and maximum likeli hood For the two class problem boosting can be viewed as an ap proximation to additive modeling on the logistic scale using maximum Bernoulli likelihood as a criterion We develop more direct approx imations and show that they exhibit nearly identical results to that of boosting Direct multi class generalizations based on multinomial likelihood are derived that exhibit performance comparable to other recently proposed multi class generalizations of boosting in most sit uations and far superior in some We suggest a minor modi cation to boosting that can reduce computation often by factors of to Finally we apply these insights to produce an alternative formu lation of boosting decision trees This approach based on best rst truncated tree induction often leads to better performance and can provide interpretable descriptions of the aggregate decision rule It is also much faster computationally making it more suitable to large scale data mining applications Department of Statistics Sequoia Hall Stanford University Stanford California fjhf trevorg stat stanford edu Department of Public Health Sciences and Department of Statistics University of Toronto tibs utstat toronto edu

Extracted Key Phrases

3 Figures and Tables

0200400'99'01'03'05'07'09'11'13'15'17
Citations per Year

5,732 Citations

Semantic Scholar estimates that this publication has 5,732 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Friedman1998AdditiveLR, title={Additive Logistic Regression a Statistical View of Boosting}, author={Jerome Friedman and Trevor Hastie and Robert Tibshirani}, year={1998} }