Language Models Learn POS First

@inproceedings{Saphra2018LanguageML,
  title={Language Models Learn POS First},
  author={Naomi Saphra and A. Lopez},
  booktitle={BlackboxNLP@EMNLP},
  year={2018}
}
  • Naomi Saphra, A. Lopez
  • Published in BlackboxNLP@EMNLP 2018
  • Computer Science
  • A glut of recent research shows that language models capture linguistic structure. Linzen et al. (2016) found that LSTM-based language models may encode syntactic information sufficient to favor verbs which match the number of their subject nouns. Liu et al. (2018) suggested that the high performance of LSTMs may depend on the linguistic structure of the input data, as performance on several artificial tasks was higher with natural language data than with artificial sequential data. Such work… CONTINUE READING

    References

    Publications referenced by this paper.
    SHOWING 1-7 OF 7 REFERENCES
    Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
    • 378
    • PDF
    Visualizing and Understanding Recurrent Networks
    • 727
    • PDF
    Learning to Generate Reviews and Discovering Sentiment
    • 282
    • PDF
    SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
    • 179
    • Highly Influential
    • PDF