What do Neural Machine Translation Models Learn about Morphology?

@inproceedings{Belinkov2017WhatDN,
  title={What do Neural Machine Translation Models Learn about Morphology?},
  author={Yonatan Belinkov and Nadir Durrani and F. Dalvi and Hassan Sajjad and James R. Glass},
  booktitle={ACL},
  year={2017}
}
  • Yonatan Belinkov, Nadir Durrani, +2 authors James R. Glass
  • Published in ACL 2017
  • Computer Science
  • Neural machine translation (MT) models obtain state-of-the-art performance while maintaining a simple, end-to-end architecture. However, little is known about what these models learn about source and target languages during the training process. In this work, we analyze the representations learned by neural MT models at various levels of granularity and empirically evaluate the quality of the representations for learning morphology through extrinsic part-of-speech and morphological tagging… CONTINUE READING
    Deep contextualized word representations
    • 4,207
    • PDF
    Synthetic and Natural Noise Both Break Neural Machine Translation
    • 241
    • PDF
    Dissecting Contextual Word Embeddings: Architecture and Representation
    • 153
    • Highly Influenced
    • PDF
    A Structural Probe for Finding Syntax in Word Representations
    • 201
    • Highly Influenced
    • PDF
    What do you learn from context? Probing for sentence structure in contextualized word representations
    • 210
    • PDF
    What Does BERT Look At? An Analysis of BERT's Attention
    • 238
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 50 REFERENCES
    Neural Machine Translation by Jointly Learning to Align and Translate
    • 12,937
    • PDF
    Adam: A Method for Stochastic Optimization
    • 50,012
    • PDF
    Sequence to Sequence Learning with Neural Networks
    • 10,547
    • PDF
    Long Short-Term Memory
    • 31,079
    • Highly Influential
    • PDF
    Neural Machine Translation of Rare Words with Subword Units
    • 2,732
    • PDF
    Character-Aware Neural Language Models
    • 1,187
    • Highly Influential
    • PDF
    Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
    • 838
    • PDF
    Exploring the Limits of Language Modeling
    • 743
    • PDF
    Character-based Neural Machine Translation
    • 218
    • PDF