Improving the Design of Induction Methods by Analyzing Algorithm Functionality and Data-Based Concept Complexity

Abstract

Although empirical machine learning has seen many algorithms, one of its most important goals has been neglected. Important real-world problems often have just a primit ive representat ion, to which the target concept bears only a remote, obscure relationship. This considerat ion leads to a class of measures that may be applied to data to estimate difficulty for standard algorithms. As the concept becomes harder, current decision tree and decision list methods give increasingly poor accuracy, though backpropagation does better. A new system for feature construction scales up best. The fundamental l imi tat ion of standard algorithms is caused by two problems: greedy search and representational inadequacy. Crit ical analysis and empirical results show that lookahead alleviates the greedy hil l-cl imbing problem at high cost, but even this is insufficient. Combining lookahead wi th feature construction alleviates the "complex global replication" problem with hard concepts. For principled algorithm development and good progress, researchers need to study hard concepts and system behavior using them.

Extracted Key Phrases

Cite this paper

@inproceedings{Rendell1993ImprovingTD, title={Improving the Design of Induction Methods by Analyzing Algorithm Functionality and Data-Based Concept Complexity}, author={Larry A. Rendell and Harish Ragavan}, booktitle={IJCAI}, year={1993} }