Improved Boosting Algorithms using Confidence-Rated Predictions

@inproceedings{Schapire1998ImprovedBA,
  title={Improved Boosting Algorithms using Confidence-Rated Predictions},
  author={Robert E. Schapire and Yoram Singer},
  booktitle={COLT},
  year={1998}
}
We describe several improvements to Freund and Schapire‘s AdaBoost boosting algorithm, particularly in a setting in which hypotheses may assign confidences to each of their predictions. [...] Key Method We give a specific method for assigning confidences to the predictions of decision trees, a method closely related to one used by Quinlan. This method also suggests a technique for growing decision trees which turns out to be identical to one proposed by Kearns and Mansour.Expand
RFBoost: An improved multi-label boosting algorithm and its application to text categorisation
TLDR
An improved version of AdaBoost.MH is proposed, called RFBoost, which proposes two methods for ranking the features: One Boosting Round and Labeled Latent Dirichlet Allocation (LLDA), a supervised topic model based on Gibbs sampling. Expand
On the Boosting Pruning Problem
TLDR
It is pointed out that the boosting pruning problem is intractable even to approximate and a margin-based theoretical heuristic is suggested for this problem. Expand
A New Boosting Algorithm Using Input-Dependent Regularizer
TLDR
Empirical studies on eight dierent UCI data sets and one text categorization data set show that WeightBoost almost always achieves a considerably better classification accuracy than AdaBoost, and experiments on data with artificially controlled noise indicate that the WeightBoost is more robust to noise than Ada boost. Expand
Adabook and Multibook - Adaptive Boosting with Chance Correction
TLDR
It is shown that the early surrender problem occurs in MultiBoost too, in multiclass situations, so that chance-corrected AdaBook and Multibook can beat standard Multiboost or AdaBoost, and the theoretical work showing the necessity of using chance- Correct measures for evaluation complements the empirical work showing how use of a chance-Corrected measure can improve boosting. Expand
Boosting with Bayesian stumps
  • D. Denison
  • Mathematics, Computer Science
  • Stat. Comput.
  • 2001
TLDR
This paper reviews some of the commonly used methods for performing boosting and shows how they can be fit into a Bayesian setup at each iteration of the algorithm. Expand
Top-Down Decision Tree Boosting and Its Applications
TLDR
By introducing a notion of pseudo-entropy functions for measuring the loss of hypotheses, this work gives a new insight into this boosting scheme from an information-theoretic viewpoint. Expand
Top-down decision tree learning as information based boosting
TLDR
A boosting technique that can be directly applied to multiclass classification problems, and it is shown that the top-down decision tree learning algorithm using the conditional G-entropy as its splitting criterion is an efficient boosting algorithm. Expand
ADABOOK & MULTIBOOK: Adaptive Boosting with Chance Correction
  • D. Powers
  • Computer Science, Mathematics
  • ArXiv
  • 2020
TLDR
It is shown that the early surrender problem occurs in MultiBoost too, in multiclass situations, so that chance-corrected AdaBook and Multibook can beat standard Multiboost or AdaBoost, and the theoretical work showing the necessity of using chance- Correct measures for evaluation complements the empirical work showing how use of a chance-Corrected measure can improve boosting. Expand
MULTIBOOK Adaptive Boosting with Chance Correction
There has been considerable interest in boosting and bagging, including the combination of the adaptive techniques of AdaBoost with the random selection with replacement techniques of Bagging. At theExpand
Learning r-of-k Functions by Boosting
TLDR
This work shows that a modification of AdaBoost (confidence-rated AdaBoost or InfoBoost) can make use of the property of r-of-k functions that make less error on one-side to find a consistent final hypothesis by using O(kr log m) hypotheses. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 27 REFERENCES
Experiments with a New Boosting Algorithm
TLDR
This paper describes experiments carried out to assess how well AdaBoost with and without pseudo-loss, performs on real learning problems and compared boosting to Breiman's "bagging" method when used to aggregate various classifiers. Expand
Using output codes to boost multiclass learning problems
TLDR
This paper describes a new technique for multiclass learning problems by combining Freund and Schapire's boosting algorithm with the main ideas of Diet- terich and Bakiri's method of error-correcting output codes (ECOC), and shows that the new hybrid method has advantages of both. Expand
Special Invited Paper-Additive logistic regression: A statistical view of boosting
Boosting is one of the most important recent developments in classification methodology. Boosting works by sequentially applying a classification algorithm to reweighted versions of the training dataExpand
A decision-theoretic generalization of on-line learning and an application to boosting
TLDR
The model studied can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting, and the multiplicative weightupdate Littlestone Warmuth rule can be adapted to this model, yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. Expand
Boosting Decision Trees
TLDR
A constructive, incremental learning system for regression problems that models data by means of locally linear experts that does not compete for data during learning and derives asymptotic results for this method. Expand
On the boosting ability of top-down decision tree learning algorithms
TLDR
This work analyzes the performance of top-down algorithms for decision tree learning and proves that some popular and empirically successful heuristics that are based on first principles meet the criteria of an independently motivated theoretical model. Expand
An Empirical Evaluation of Bagging and Boosting
TLDR
The results clearly show that even though Bagging almost always produces a better classifier than any of its individual component classifiers and is relatively impervious to overfitting, it does not generalize any better than a baseline neural-network ensemble method. Expand
Using and combining predictors that specialize
TLDR
It is shown how to transform algorithms that assume that all experts are always awake to algorithms that do not require this assumption, and how to derive corresponding loss bounds. Expand
Arcing Classifiers
Recent work has shown that combining multiple versions of unstable classifiers such as trees or neural nets results in reduced test set error. One of the more effective is bagging (Breiman [1996a])Expand
Arcing classifier (with discussion and a rejoinder by the author)
Recent work has shown that combining multiple versions of unstable classifiers such as trees or neural nets results in reduced test set error. One of the more effective is bagging. Here, modifiedExpand
...
1
2
3
...