Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies

@article{Linzen2016AssessingTA,
  title={Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies},
  author={Tal Linzen and Emmanuel Dupoux and Yoav Goldberg},
  journal={Transactions of the Association for Computational Linguistics},
  year={2016},
  volume={4},
  pages={521-535}
}
The success of long short-term memory (LSTM) neural networks in language processing is typically attributed to their ability to capture long-distance statistical regularities. Linguistic regularities are often sensitive to syntactic structure; can such dependencies be captured by LSTMs, which do not have explicit structural representations? We begin addressing this question using number agreement in English subject-verb dependencies. We probe the architecture’s grammatical competence both using… CONTINUE READING

Topics from this paper.

Citations

Publications citing this paper.
SHOWING 1-10 OF 221 CITATIONS

Finding Syntactic Representations in Neural Stacks

VIEW 8 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

The emergence of number and syntax units in LSTM language models

  • NAACL-HLT
  • 2019
VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Analysis Methods in Neural Language Processing: A Survey

  • Transactions of the Association for Computational Linguistics
  • 2018
VIEW 5 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

Can LSTM Learn to Capture Agreement? The Case of Basque

VIEW 11 EXCERPTS
CITES BACKGROUND, METHODS & RESULTS
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2016
2019

CITATION STATISTICS

  • 30 Highly Influenced Citations

  • Averaged 73 Citations per year from 2017 through 2019

References

Publications referenced by this paper.
SHOWING 1-10 OF 51 REFERENCES

The Cambridge Grammar of the English Language

VIEW 9 EXCERPTS
HIGHLY INFLUENTIAL

Finding Structure in Time

  • Cognitive Science
  • 1990
VIEW 11 EXCERPTS
HIGHLY INFLUENTIAL

Visualizing and Understanding Recurrent Networks

  • ArXiv
  • 2015
VIEW 3 EXCERPTS
HIGHLY INFLUENTIAL

Agreement with nearest always bad? http://itre.cis.upenn.edu/ ̃myl/ languagelog/archives/001846.html

Arnold Zwicky
  • 2005
VIEW 9 EXCERPTS
HIGHLY INFLUENTIAL