Corpus ID: 12926428

Potential Boosters?

  title={Potential Boosters?},
  author={Nigel P. Duffy and David P. Helmbold},
Recent interpretations of the Adaboost algorithm view it as performing a gradient descent on a potential function. Simply changing the potential function allows one to create new algorithms related to AdaBoost. However, these new algorithms are generally not known to have the formal boosting property. This paper examines the question of which potential functions lead to new algorithms that are boosters. The two main results are general sets of conditions on the potential; one set implies that… Expand
A geometric approach to leveraging weak learners
A new leveraging algorithm is introduced based on a natural potential function for improving the hypotheses generated by weak learning algorithms and is likely to perform better than AdaBoost on noisy data and with weak learners returning low confidence hypotheses. Expand
Robust boosting and its relation to bagging
  • S. Rosset
  • Mathematics, Computer Science
  • KDD '05
  • 2005
An approach of weight decay for observation weights which is equivalent to "robustifying" the underlying loss function is presented which converges to Bagging, which can be viewed as boosting with a linear underlying lossfunction. Expand
The Synergy Between PAV and AdaBoost
It is shown how PAV may be applied to a weak hypothesis to yield a new weak hypothesis which is in a sense an ideal confidence rated prediction and that this leads to an optimal updating for AdaBoost. Expand
Logistic Regression, AdaBoost and Bregman Distances
A unified account of boosting and logistic regression in which each learning problem is cast in terms of optimization of Bregman distances, and a parameterized family of algorithms that includes both a sequential- and a parallel-update algorithm as special cases are described, thus showing how the sequential and parallel approaches can themselves be unified. Expand
Boosting and Maximum Likelihood for Exponential Models
We derive an equivalence between AdaBoost and the dual of a convex optimization problem, showing that the only difference between minimizing the exponential loss used by AdaBoost and maximumExpand
Advance and Prospects of AdaBoost Algorithm
Abstract AdaBoost is one of the most excellent Boosting algorithms. It has a solid theoretical basis and has made great success in practical applications. AdaBoost can boost a weak learning algorithmExpand
Generalized Boosting Algorithms for Convex Optimization
This work analyzes gradient-based descent algorithms for boosting with respect to any convex objective and introduces a new measure of weak learner performance into this setting which generalizes existing work. Expand
Barrier Boosting
It is shown that convergence of Boosting-type algorithms becomes simpler to prove and directions to develop further Boosting schemes are outlined, in particular a new Boosting technique for regression – "-Boost – is proposed. Expand
Boosting Methods for Regression
This paper examines ensemble methods for regression that leverage or “boost” base regressors by iteratively calling them on modified samples and bound the complexity of the regression functions produced in order to derive PAC-style bounds on their generalization errors. Expand
Random classification noise defeats all convex potential boosters
A broad class of boosting algorithms can be interpreted as performing coordinate-wise gradient descent to minimize some potential function of the margins of a data set to avoid random classification noise. Expand


A Geometric Approach to Leveraging Weak Learners
A new leveraging algorithm is introduced based on a natural potential function that has bounds that are incomparable to AdaBoost's, and their empirical performance is similar to Ada boost's. Expand
Boosting Algorithms as Gradient Descent
Following previous theoretical results bounding the generalization performance of convex combinations of classifiers in terms of general cost functions of the margin, a new algorithm (DOOM II) is presented for performing a gradient descent optimization of such cost functions. Expand
Boosting as entropy projection
It is shown how AdaBoost’s choice of the new distribution can be seen as an approximate solution to the following problem: Find a new distribution that is closest to the old distribution subject to the constraint that thenew distribution is orthogonal to the vector of mistakes of the current weak hypothesis. Expand
Improved Boosting Algorithms using Confidence-Rated Predictions
We describe several improvements to Freund and Schapire‘s AdaBoost boosting algorithm, particularly in a setting in which hypotheses may assign confidences to each of their predictions. We give aExpand
Additive models, boosting, and inference for generalized divergences
A framework for designing incremental learning algorithms derived from generalized entropy functionals based on the use of Bregman divergences together with the associated class of additive models constructed using the Legendre transform is presented. Expand
A decision-theoretic generalization of on-line learning and an application to boosting
The model studied can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting, and the multiplicative weightupdate Littlestone Warmuth rule can be adapted to this model, yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. Expand
Arcing the edge
Recent work has shown that adaptively reweighting the training set, growing a classifier using the new weights, and combining the classifiers constructed to date can significantly decreaseExpand
Boosting the margin: A new explanation for the effectiveness of voting methods
It is shown that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. Expand
Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By
The main and important contribution of this paper is in establishing a connection between boosting, a newcomer to the statistics scene, and additive models. One of the main properties of boostingExpand
Design and analysis of efficient learning algorithms
  • R. Schapire
  • Computer Science
  • ACM Doctoral dissertation award ; 1991
  • 1992
It is shown that any "weak" learning algorithm that performs just slightly better than random guessing can be converted into one whose error can be made arbitrarily small, and a technique for converting any PAC-learning algorithm into one that is highly space efficient is explored. Expand