On Evaluating the Generalization of LSTM Models in Formal Languages
@inproceedings{Paulson2018OnET, title={On Evaluating the Generalization of LSTM Models in Formal Languages}, author={John A. Paulson}, year={2018} }
Recurrent Neural Networks (RNNs) are theoretically Turing-complete and established themselves as a dominant model for language processing. Yet, there still remains an uncertainty regarding their language learning capabilities. In this paper, we empirically evaluate the inductive learning capabilities of Long Short-Term Memory networks, a popular extension of simple RNNs, to learn simple formal languages, in particular ab, abc, and abcd. We investigate the influence of various aspects of… CONTINUE READING
Figures and Tables from this paper
References
SHOWING 1-10 OF 25 REFERENCES
On the Practical Computational Power of Finite Precision RNNs for Language Recognition
- Computer Science, Mathematics
- ACL
- 2018
- 109
- Highly Influential
- PDF
LSTM recurrent networks learn simple context-free and context-sensitive languages
- Computer Science, Medicine
- IEEE Trans. Neural Networks
- 2001
- 477
- PDF
LSTM: A Search Space Odyssey
- Computer Science, Medicine
- IEEE Transactions on Neural Networks and Learning Systems
- 2017
- 2,336
- PDF
Incremental training of first order recurrent neural networks to predict a context-sensitive language
- Computer Science, Medicine
- Neural Networks
- 2003
- 35
- PDF
A Recurrent Neural Network that Learns to Count
- Computer Science
- Connect. Sci.
- 1999
- 202
- Highly Influential
- PDF