The Random Subspace Method for Constructing Decision Forests

@article{Ho1998TheRS,
  title={The Random Subspace Method for Constructing Decision Forests},
  author={T. Ho},
  journal={IEEE Trans. Pattern Anal. Mach. Intell.},
  year={1998},
  volume={20},
  pages={832-844}
}
  • T. Ho
  • Published 1998
  • Mathematics, Computer Science
  • IEEE Trans. Pattern Anal. Mach. Intell.
Much of previous attention on decision trees focuses on the splitting criteria and optimization of tree sizes. [...] Key Method The classifier consists of multiple trees constructed systematically by pseudorandomly selecting subsets of components of the feature vector, that is, trees constructed in randomly chosen subspaces.Expand

Paper Mentions

Random Forests with Stochastic Induction of Decision Trees
TLDR
The proposed algorithm is based on a stochastic process to induct each decision tree, assigning a probability for the selection of the split attribute in every tree node, designed in order to create strong and independent trees. Expand
On Dynamic Selection of Subspace for Random Forest
TLDR
A new technique is proposed that randomly determines the size of the attribute subset between a dynamically determined range based on the relative size of current data segment to the bootstrap samples at each node splitting event in Random Forest. Expand
Building forests of local trees
TLDR
A novel algorithm for ensemble classifiers is proposed, in which ensemble components are trained with focus on different regions of the sample space, which highlights that these ensembles perform better than other relevant kinds of ensemble classifier classifiers, including random forests. Expand
A local complexity based combination method for decision forests trained with high-dimensional data
TLDR
A competence estimate is proposed which is based on local complexity measurements, to perform a weighted average combination of the decision forest and significantly outperforms the standard non-weighted average combination and also the renowned Classifier Local Accuracy competence estimate, while consuming significantly less time. Expand
Towards generating random forests via extremely randomized trees
TLDR
The results on several public datasets show that random partition without exhaustive search at each node of a decision tree can yield better performance with less computational complexity. Expand
Decision tree ensemble construction incorporating feature values modification and random subspace method
TLDR
The proposed hybrid ensemble method was evaluated on several benchmark classification problems, and was found to achieve performance better than or competitive with related conventional methods. Expand
Improved Threshold Selection by Using Calibrated Probabilities for Random Forest Classifiers
TLDR
This paper proposes to introduce an additional condition for selecting the best variable leading to an improvement of the recognition accuracy, especially for a smaller number of trees, and enhances the standard threshold selection by a quality estimation that is computed using a calibration method. Expand
2 Three Techniques to Construct Rough Decision Forests
Decision forests are a type of classification paradigm which combines a collection of decision trees for a classification task, instead of depending on a single tree. Improvement of accuracy andExpand
Constructing Rough Decision Forests
TLDR
Three methods to select decision trees or reducts are presented, and decisions from selected trees are fused with the plurality voting rule, and the experiments show that random selection is the worst solution in the proposed methods. Expand
Classification by ensembles from random partitions of high-dimensional data
TLDR
This study shows that the proposed methods perform consistently well compared to widely used classification methods, and maintains the balance between sensitivity and specificity more adequately than many other classification methods considered in this study. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 32 REFERENCES
C4.5 decision forests
  • T. Ho
  • Mathematics, Computer Science
  • Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170)
  • 1998
TLDR
This work proposes a method to construct a decision tree based classifier that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity. Expand
Structure-driven induction of decision tree classifiers through neural learning
TLDR
A structure-driven approach for induction of decision trees, SDIDT, is proposed, which generates compact trees that have multifeature splits at each internal node which are determined on global rather than local basis; consequently it produces decision trees yielding better classification and interpretation of the underlying relationships in the data. Expand
Induction of Oblique Decision Trees
TLDR
A randomized technique for partitioning examples using oblique hyperplanes to create smaller but equally accurate decision trees than those created by other methods, and is found to produce surprisingly small trees without losing predic-tive accuracy. Expand
Multiple binary decision tree classifiers
  • S. Shlien
  • Mathematics, Computer Science
  • Pattern Recognit.
  • 1990
TLDR
This paper demonstrates that higher classification accuracies can be obtained from the same training set by using a combination of decision trees and by reaching a consensus using Dempster and Shafer's theory of evidence. Expand
Multiple decision trees
TLDR
This paper describes experiments, on two domains, to investigate the effect of averaging over predictions of multiple decision trees, instead of using a single tree, finding that it is best to average across sets of trees with different structure; this usually gives better performance than any of the constituent trees, including the ID3 tree. Expand
A System for Induction of Oblique Decision Trees
TLDR
This system, OC1, combines deterministic hill-climbing with two forms of randomization to find a good oblique split (in the form of a hyperplane) at each node of a decision tree. Expand
Decision Combination in Multiple Classifier Systems
TLDR
This work proposes three methods based on the highest rank, the Borda count, and logistic regression for class set reranking that have been tested in applications of degraded machine-printed characters and works from large lexicons, resulting in substantial improvement in overall correctness. Expand
Bagging, Boosting, and C4.5
TLDR
Results of applying Breiman's bagging and Freund and Schapire's boosting to a system that learns decision trees and testing on a representative collection of datasets show boosting shows the greater benefit. Expand
Experiments with a New Boosting Algorithm
TLDR
This paper describes experiments carried out to assess how well AdaBoost with and without pseudo-loss, performs on real learning problems and compared boosting to Breiman's "bagging" method when used to aggregate various classifiers. Expand
Chapter 18 Committees of decision trees
Abstract Many intelligent systems are designed to sift through a mass of evidence and arrive at a decision. Certain pieces of evidence may be given more weight than others, and this may affect theExpand
...
1
2
3
4
...