Rational Recurrences

@article{Peng2018RationalR,
  title={Rational Recurrences},
  author={Hao Peng and Roy Schwartz and Sam Thomson and Noah A. Smith},
  journal={ArXiv},
  year={2018},
  volume={abs/1808.09357}
}
Despite the tremendous empirical success of neural models in natural language processing, many of them lack the strong intuitions that accompany classical machine learning approaches. [...] Key Method We characterize this connection formally, defining rational recurrences to be recurrent hidden state update functions that can be written as the Forward calculation of a finite set of WFSAs. We show that several recent neural models use rational recurrences. Our analysis provides a fresh view of these models and…Expand
22 Citations
Sequential Neural Networks as Automata
  • 25
  • PDF
Sequential Neural Networks as Automata Anonymous ACL submission
  • PDF
RNN Architecture Learning with Sparse Regularization
  • 5
  • PDF
A Lightweight Recurrent Network for Sequence Modeling
  • 1
  • PDF
A Lightweight Recurrent Network for Sequence Modeling
  • 2
  • PDF
Cold-start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks
  • PDF
Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)
  • 29
  • PDF
Connecting Weighted Automata and Recurrent Neural Networks through Spectral Learning
  • 21
  • PDF
...
1
2
3
...

References

SHOWING 1-10 OF 70 REFERENCES
Strongly-Typed Recurrent Neural Networks
  • 43
  • PDF
Neural Architecture Search with Reinforcement Learning
  • 2,443
  • PDF
Learning Longer Memory in Recurrent Neural Networks
  • 195
  • Highly Influential
  • PDF
SoPa: Bridging CNNs, RNNs, and Weighted Finite-State Machines
  • 8
  • PDF
Recurrent Neural Networks as Weighted Language Recognizers
  • 31
  • PDF
On the State of the Art of Evaluation in Neural Language Models
  • 345
  • PDF
A Primer on Neural Network Models for Natural Language Processing
  • 653
  • Highly Influential
  • PDF
Intelligible Language Modeling with Input Switched Affine Networks
  • 7
  • PDF
Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
  • 473
Attention is All you Need
  • 17,572
  • PDF
...
1
2
3
4
5
...