# Potential Boosters?

@inproceedings{Duffy1999PotentialB, title={Potential Boosters?}, author={Nigel P. Duffy and D. Helmbold}, booktitle={NIPS}, year={1999} }

Recent interpretations of the Adaboost algorithm view it as performing a gradient descent on a potential function. Simply changing the potential function allows one to create new algorithms related to AdaBoost. However, these new algorithms are generally not known to have the formal boosting property. This paper examines the question of which potential functions lead to new algorithms that are boosters. The two main results are general sets of conditions on the potential; one set implies that… Expand

#### Topics from this paper

#### 43 Citations

A geometric approach to leveraging weak learners

- Computer Science, Mathematics
- Theor. Comput. Sci.
- 2002

A new leveraging algorithm is introduced based on a natural potential function for improving the hypotheses generated by weak learning algorithms and is likely to perform better than AdaBoost on noisy data and with weak learners returning low confidence hypotheses. Expand

Robust boosting and its relation to bagging

- Mathematics, Computer Science
- KDD '05
- 2005

An approach of weight decay for observation weights which is equivalent to "robustifying" the underlying loss function is presented which converges to Bagging, which can be viewed as boosting with a linear underlying lossfunction. Expand

The Synergy Between PAV and AdaBoost

- Mathematics, Medicine
- Machine Learning
- 2005

It is shown how PAV may be applied to a weak hypothesis to yield a new weak hypothesis which is in a sense an ideal confidence rated prediction and that this leads to an optimal updating for AdaBoost. Expand

Logistic Regression, AdaBoost and Bregman Distances

- Mathematics, Computer Science
- Machine Learning
- 2004

A unified account of boosting and logistic regression in which each learning problem is cast in terms of optimization of Bregman distances, and a parameterized family of algorithms that includes both a sequential- and a parallel-update algorithm as special cases are described, thus showing how the sequential and parallel approaches can themselves be unified. Expand

Boosting and Maximum Likelihood for Exponential Models

- Computer Science, Mathematics
- NIPS
- 2001

We derive an equivalence between AdaBoost and the dual of a convex optimization problem, showing that the only difference between minimizing the exponential loss used by AdaBoost and maximum… Expand

Advance and Prospects of AdaBoost Algorithm

- Mathematics
- 2013

Abstract AdaBoost is one of the most excellent Boosting algorithms. It has a solid theoretical basis and has made great success in practical applications. AdaBoost can boost a weak learning algorithm… Expand

Generalized Boosting Algorithms for Convex Optimization

- Mathematics, Computer Science
- ICML
- 2011

This work analyzes gradient-based descent algorithms for boosting with respect to any convex objective and introduces a new measure of weak learner performance into this setting which generalizes existing work. Expand

Barrier Boosting

- Computer Science
- COLT
- 2000

It is shown that convergence of Boosting-type algorithms becomes simpler to prove and directions to develop further Boosting schemes are outlined, in particular a new Boosting technique for regression – "-Boost – is proposed. Expand

Boosting Methods for Regression

- Mathematics, Computer Science
- Machine Learning
- 2004

This paper examines ensemble methods for regression that leverage or “boost” base regressors by iteratively calling them on modified samples and bound the complexity of the regression functions produced in order to derive PAC-style bounds on their generalization errors. Expand

Random classification noise defeats all convex potential boosters

- Mathematics, Computer Science
- ICML
- 2008

A broad class of boosting algorithms can be interpreted as performing coordinate-wise gradient descent to minimize some potential function of the margins of a data set to avoid random classification noise. Expand

#### References

SHOWING 1-10 OF 28 REFERENCES

A Geometric Approach to Leveraging Weak Learners

- Computer Science
- EuroCOLT
- 1999

A new leveraging algorithm is introduced based on a natural potential function that has bounds that are incomparable to AdaBoost's, and their empirical performance is similar to Ada boost's. Expand

Boosting Algorithms as Gradient Descent

- Computer Science
- NIPS
- 1999

Following previous theoretical results bounding the generalization performance of convex combinations of classifiers in terms of general cost functions of the margin, a new algorithm (DOOM II) is presented for performing a gradient descent optimization of such cost functions. Expand

Boosting as entropy projection

- Mathematics, Computer Science
- COLT '99
- 1999

It is shown how AdaBoost’s choice of the new distribution can be seen as an approximate solution to the following problem: Find a new distribution that is closest to the old distribution subject to the constraint that thenew distribution is orthogonal to the vector of mistakes of the current weak hypothesis. Expand

Improved Boosting Algorithms using Confidence-Rated Predictions

- Mathematics, Computer Science
- COLT
- 1998

We describe several improvements to Freund and Schapire‘s AdaBoost boosting algorithm, particularly in a setting in which hypotheses may assign confidences to each of their predictions. We give a… Expand

Additive models, boosting, and inference for generalized divergences

- Mathematics, Computer Science
- COLT '99
- 1999

A framework for designing incremental learning algorithms derived from generalized entropy functionals based on the use of Bregman divergences together with the associated class of additive models constructed using the Legendre transform is presented. Expand

A decision-theoretic generalization of on-line learning and an application to boosting

- Computer Science
- EuroCOLT
- 1995

The model studied can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting, and the multiplicative weightupdate Littlestone Warmuth rule can be adapted to this model, yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. Expand

Arcing the edge

- 1997

Recent work has shown that adaptively reweighting the training set, growing a classifier using the new weights, and combining the classifiers constructed to date can significantly decrease… Expand

Boosting the margin: A new explanation for the effectiveness of voting methods

- Mathematics, Computer Science
- ICML
- 1997

It is shown that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. Expand

Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By

- 2000

The main and important contribution of this paper is in establishing a connection between boosting, a newcomer to the statistics scene, and additive models. One of the main properties of boosting… Expand

Design and analysis of efficient learning algorithms

- Computer Science
- ACM Doctoral dissertation award ; 1991
- 1992

It is shown that any "weak" learning algorithm that performs just slightly better than random guessing can be converted into one whose error can be made arbitrarily small, and a technique for converting any PAC-learning algorithm into one that is highly space efficient is explored. Expand