Corpus ID: 208162197

On Evaluating the Generalization of LSTM Models in Formal Languages

  title={On Evaluating the Generalization of LSTM Models in Formal Languages},
  author={John A. Paulson},
  • John A. Paulson
  • Published 2018
  • Recurrent Neural Networks (RNNs) are theoretically Turing-complete and established themselves as a dominant model for language processing. Yet, there still remains an uncertainty regarding their language learning capabilities. In this paper, we empirically evaluate the inductive learning capabilities of Long Short-Term Memory networks, a popular extension of simple RNNs, to learn simple formal languages, in particular ab, abc, and abcd. We investigate the influence of various aspects of… CONTINUE READING

    Figures and Tables from this paper


    LSTMs Exploit Linguistic Attributes of Data
    • 13
    • PDF
    On the Practical Computational Power of Finite Precision RNNs for Language Recognition
    • 109
    • Highly Influential
    • PDF
    LSTM recurrent networks learn simple context-free and context-sensitive languages
    • 477
    • PDF
    LSTM: A Search Space Odyssey
    • 2,336
    • PDF
    Tail-recursive Distributed Representations and Simple Recurrent Networks
    • 39
    A Recurrent Network that performs a Context-Sensitive Prediction Task
    • 39
    • PDF
    Sequence to Sequence Learning with Neural Networks
    • 11,464
    • PDF
    A Recurrent Neural Network that Learns to Count
    • 202
    • Highly Influential
    • PDF