Extracting Knowledge from Neural Networks

Abstract

IntroductIon Neural networks (NN) as classifier systems have shown great promise in many problem domains in empirical studies over the past two decades. Using case classification accuracy as the criteria, neural networks have typically outperformed traditional parametric techniques (e.g., discriminant analysis, logistic regression) as well as other non-paramet-ric approaches (e.g., various inductive learning systems such as ID3, C4.5, CART, etc.). In spite of this strong evidence of superior performance, the use of neural networks in organizations has been hampered by the lack of an " easy " way of explaining what the neural network has learned about the domain being studied. It is well known that knowledge in a neural network is " mysteriously " encapsulated in its connection weights. It is well accepted that decision-makers prefer techniques that can provide good explanations about the knowledge found in a domain even if they are less effective in terms of classification accuracy. Over the past decade, neural network researchers have thus begun an active research stream that focuses on developing techniques for extracting usable knowledge from a trained neural network. The literature has become quite vast and, unfortunately , still lacks any form of consensus on the best way to help neural networks be more useful to knowledge discovery practitioners. This article will then provide a brief review of recent work in one specific area of the neural net-work/knowledge discovery research stream. This review considers knowledge extraction techniques that create IF-THEN rules from trained feed-forward neural networks used as classifiers. We chose this narrow view for a couple of important reasons. First, as mentioned, the research in this area is extraordinarily broad and a critical review cannot be done without focusing on a smaller subset within the literature. Second, 749 Extracting Knowledge from Neural Networks classification problems are a familiar problem in business. Third, creating basic IF-THEN rules from a trained neural network is viewed as the most useful area in the entire research stream for the knowledge management and data mining practitioner. With this narrow focus, some aspects of knowledge extraction from neural networks are obviously not mentioned here. With the focus on deterministic IF-THEN rules, outputs that include " fuzziness " (fuzzy logic) are omitted. In addition, research that involves different neural network architectures (e.g., recurrent networks) and/or different knowledge discovery problem areas (e.g., regression/prediction rather than classification) are also excluded from the review.

Extracted Key Phrases

1 Figure or Table

Cite this paper

@inproceedings{Fuller2011ExtractingKF, title={Extracting Knowledge from Neural Networks}, author={Christie M. Fuller and Rick L. Wilson}, booktitle={Encyclopedia of Knowledge Management}, year={2011} }