Solving Multiclass Learning Problems via Error-Correcting Output Codes
@article{Dietterich1994SolvingML, title={Solving Multiclass Learning Problems via Error-Correcting Output Codes}, author={Thomas G. Dietterich and Ghulum Bakiri}, journal={J. Artif. Intell. Res.}, year={1994}, volume={2}, pages={263-286} }
Multiclass learning problems involve finding a definition for an unknown function f(x) whose range is a discrete set containing k > 2 values (i.e., k "classes"). The definition is acquired by studying collections of training examples of the form (xi, f(xi)). Existing approaches to multiclass learning problems include direct application of multiclass algorithms such as the decision-tree algorithms C4.5 and CART, application of binary concept learning algorithms to learn individual binary…
Figures and Tables from this paper
2,960 Citations
On the Consistency of Output Code Based Learning Algorithms for Multiclass Learning Problems
- Computer ScienceCOLT
- 2014
This is the first work that comprehensively studies consistency properties of output code based methods for multiclass learning, and derives general conditions on the binary surrogate loss under which the one-vs-all and all-pairs code matrices yield consistent algorithms with respect to the multiclass 0-1 loss.
Using output codes to boost multiclass learning problems
- Computer ScienceICML
- 1997
This paper describes a new technique for multiclass learning problems by combining Freund and Schapire's boosting algorithm with the main ideas of Diet- terich and Bakiri's method of error-correcting output codes (ECOC), and shows that the new hybrid method has advantages of both.
evolutionary Design of Code-matrices for Multiclass Problems
- Computer ScienceSoft Computing for Knowledge Discovery and Data Mining
- 2008
This chapter presents a survey on techniques for multiclass problems code-matrix design, and shows how evolutionary techniques can be employed to solve this problem.
Stochastic Organization of Output Codes in Multiclass Learning Problems
- Computer ScienceNeural Computation
- 2001
This work presents a novel algorithm that applies a maximum-likelihood objective function in conjunction with the expectation-maximization (EM) algorithm, and shows the potential gain of the optimized output codes over OPC or ECOC methods.
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers
- Computer ScienceJ. Mach. Learn. Res.
- 2000
A general method for combining the classifiers generated on the binary problems is proposed, and a general empirical multiclass loss bound is proved given the empirical loss of the individual binary learning algorithms.
Multiclass boosting with repartitioning
- Computer ScienceICML
- 2006
This paper proposes a new multiclass boosting algorithm that modifies the coding matrix according to the learning ability of the base learner, and shows experimentally that this algorithm is very efficient in optimizing the multiclass margin cost, and outperforms existing multiclass algorithms such as AdaBoost.
Multiclass learning, boosting, and error-correcting codes
- Computer ScienceCOLT '99
- 1999
ECC, that, by using a different weighting of the votes of the weak hypotheses, is able to improve on the performance of ADABOOST.OC, is arguably a more direct reduction of multiclass learning to binary learning problems than previous multiclass boosting algorithms.
Adaptive Error-Correcting Output Codes
- Computer ScienceIJCAI
- 2013
This paper reformulate the ECOC models from the perspective of multi-task learning, where the binary classifiers are learned in a common subspace of data, and presents the kernel extension of the proposed model.
CCE: An ensemble architecture based on coupled ANN for solving multiclass problems
- Computer ScienceInf. Fusion
- 2020
References
SHOWING 1-10 OF 33 REFERENCES
FUNCTION MODELING EXPERIMENTS.
- Computer Science
- 1963
The results of an experimental investigation of the capabilities and the limitations of trainable machines for use in function modeling have been presented, finding that for more difficult applications, the machine performance was sufficiently good to make the speed advantages of a training machine a significant consideration.
Why Error Correcting Output Coding Works
- Computer Science
- 1994
An empirical investigation of why the ECOC technique works particularly when employed with decision tree learning methods concludes that an important factor in the success of the method is the nearly random behavior of decision tree algorithms near the root of the decision tree when applied to learn decision boundaries.
An improved boosting algorithm and its implications on learning complexity
- Computer ScienceCOLT '92
- 1992
The main result is an improvement of the boosting-by-majority algorithm, which shows that the majority rule is the optimal rule for combining general weak learners and extends the boosting algorithm to concept classes that give multi-valued labels and real-valuedlabel.
Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters
- Computer ScienceNIPS
- 1989
It is shown that once the output layer of a multilayer perceptron is modified to provide mathematically correct probability distributions, and the usual squared error criterion is replaced with a probability-based score, the result is equivalent to Maximum Mutual Information training.
When Networks Disagree: Ensemble Methods for Hybrid Neural Networks
- Computer Science
- 1992
Experimental results show that the ensemble method dramatically improves neural network performance on difficult real-world optical character recognition tasks.
Neural Network Classifiers Estimate Bayesian a posteriori Probabilities
- Computer ScienceNeural Computation
- 1991
Results of Monte Carlo simulations performed using multilayer perceptron (MLP) networks trained with backpropagation, radial basis function (RBF) networks, and high-order polynomial networks graphically demonstrate that network outputs provide good estimates of Bayesian probabilities.
Classification and regression trees
- Computer Science
An introduction to classification and regression trees is given by reviewing some widely available algorithms and comparing their capabilities, strengths, and weakness in two examples.
Backpropagation Applied to Handwritten Zip Code Recognition
- Computer ScienceNeural Computation
- 1989
This paper demonstrates how constraints from the task domain can be integrated into a backpropagation network through the architecture of the network, successfully applied to the recognition of handwritten zip code digits provided by the U.S. Postal Service.