# A Comparative Study of Rule Extraction for Recurrent Neural Networks

@article{Wang2018ACS, title={A Comparative Study of Rule Extraction for Recurrent Neural Networks}, author={Qinglong Wang and Kaixuan Zhang and Alexander Ororbia and Xinyu Xing and Xue Liu and C. Lee Giles}, journal={arXiv: Learning}, year={2018} }

Understanding recurrent networks through rule extraction has a long history. This has taken on new interests due to the need for interpreting or verifying neural networks. One basic form for representing stateful rules is deterministic finite automata (DFA). Previous research shows that extracting DFAs from trained second-order recurrent networks is not only possible but also relatively stable. Recently, several new types of recurrent networks with more complicated architectures have been…

## 11 Citations

### Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks

- Computer ScienceAAAI
- 2021

In this paper, a novel approach to extracting weighted automata with the guidance of a target RNN's decision and context information is proposed and a state composition method is proposed to enhance the context-awareness of the extracted model.

### State-Regularized Recurrent Neural Networks

- Computer ScienceICML
- 2019

It is shown that state-regularization simplifies the extraction of finite state automata modeling an RNN's state transition dynamics and forces RNNs to operate more like automata with external memory and less like finite state machines, which makes Rnns have better interpretability and explainability.

### Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecture

- Computer ScienceKnowl. Based Syst.
- 2022

### On the Ability and Limitations of Transformers to Recognize Formal Languages

- Computer ScienceEMNLP
- 2020

This work systematically study the ability of Transformers to model such languages as well as the role of its individual components in doing so, and provides insights on therole of self-attention mechanism in modeling certain behaviors and the influence of positional encoding schemes on the learning and generalization abilities.

### Learning With Interpretable Structure From Gated RNN

- Computer ScienceIEEE Transactions on Neural Networks and Learning Systems
- 2020

Finite-state automaton (FSA) that processes sequential data have a more interpretable inner mechanism according to the definition of interpretability and can be learned from RNNs as the interpretable structure, and FSA is more trustable than the RNN from which it learned.

### Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions

- Computer ScienceArXiv
- 2022

An extensive empirical study on Boolean functions provides strong quantiﬁable evidence that suggestserences in the inductive biases of Transformers and recurrent models which may help explain Transformer’s generalization performance despite relatively limited expressiveness.

### Adversarial Models for Deterministic Finite Automata

- Computer ScienceCanadian Conference on AI
- 2020

This work investigates a finer-grained understanding of the characteristics of particular deterministic finite automata (DFA) and proposes an adversarial model that reveals the sensitive transitions embedded in a DFA.

### A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts

- Computer ScienceData Mining and Knowledge Discovery
- 2023

A complete taxonomy of XAI methods with respect to notions present in the current state of research is provided and provides foundations for targeted, use-case-oriented, and context-sensitive future research.

### Shapley Homology: Topological Analysis of Sample Influence for Neural Networks

- Computer ScienceNeural Computation
- 2020

The Shapley homology framework is proposed, which provides a quantitative metric for the influence of a sample of the homology of a simplicial complex and shows that samples with higher influence scores have a greater impact on the accuracy of neural networks that determine graph connectivity and on several regular grammars whose higher entropy values imply greater difficulty in being learned.

### XAI Method Properties: A (Meta-)study

- Computer ScienceArXiv
- 2021

This paper summarizes the most cited and current taxonomies in a meta-analysis in order to highlight the essential aspects of the state-of-the-art in XAI.

## References

SHOWING 1-10 OF 50 REFERENCES

### Rule Extraction from Recurrent Neural Networks: ATaxonomy and Review

- Computer ScienceNeural Computation
- 2005

This article reviews the progress of techniques for extraction of rules from RNNs and develops a taxonomy specifically designed for this purpose, and identifies important open research issues that can give the field a significant push forward.

### Symbolic Knowledge Representation in Recurrent Neural Networks: Insights from Theoretical Models of

- Computer Science
- 2000

This chapter addresses some fundamental issues in regard to recurrent neural network architectures and learning algorithms, their computational power, their suitability for diierent classes of applications, and their ability to acquire symbolic knowledge through learning.

### Inducing Regular Grammars Using Recurrent Neural Networks

- Computer ScienceArXiv
- 2017

This work trains a recurrent neural network to distinguish between strings that are in or outside a regular language, and utilizes an algorithm for extracting the learned finite-state automaton.

### Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks

- Computer ScienceNeural Computation
- 1992

It is shown that a recurrent, second-order neural network using a real-time, forward training algorithm readily learns to infer small regular grammars from positive and negative string training samples, and many of the neural net state machines are dynamically stable, that is, they correctly classify many long unseen strings.

### An Empirical Evaluation of Recurrent Neural Network Rule Extraction

- Computer ScienceArXiv
- 2017

It is shown that production rules can be stably extracted from trained RNNs and that in certain cases the rules outperform the trained Rnns.

### Second-order recurrent neural networks for grammatical inference

- Computer ScienceIJCNN-91-Seattle International Joint Conference on Neural Networks
- 1991

It is shown that a recurrent, second-order neural network using a real-time, feedforward training algorithm readily learns to infer regular grammars from positive and negative string training samples and many of the neural net state machines are dynamically stable and correctly classify long unseen strings.

### An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks

- Computer ScienceNeural Computation
- 2018

It is shown that production rules can be stably extracted from trained RNNs and that in certain cases, the rules outperform the trained Rnns.

### Learning long-term dependencies in NARX recurrent neural networks

- Computer ScienceIEEE Trans. Neural Networks
- 1996

It is shown that the long-term dependencies problem is lessened for a class of architectures called nonlinear autoregressive models with exogenous (NARX) recurrent neural networks, which have powerful representational capabilities.

### The induction of dynamical recognizers

- Computer ScienceMachine Learning
- 2004

A longitudinal examination of the learning process of a higher order recurrent neural network architecture illustrates a new form of mechanical inference: Induction by phase transition, and a hypothesis relating linguistic generative capacity to the behavioral regimes of non-linear dynamical systems is concluded.

### Active Grammatical Inference: A New Learning Methodology

- Computer Science
- 1994

A new methodology for inferring regular grammars y from a set of positive and negative examples and a-priori knowledge (if available) is presented, that is based on a combination of neural and symbolic techniques, which can be guided by the validated previous results and/or by introducing constraints or (positive/negative) rules dynamically.