Long-Distance Dependencies Don't Have to Be Long: Simplifying through Provably (Approximately) Optimal Permutations

  title={Long-Distance Dependencies Don't Have to Be Long: Simplifying through Provably (Approximately) Optimal Permutations},
  author={Rishi Bommasani},
  • Rishi Bommasani
  • Published in ACL 2019
  • Computer Science
  • Neural models at the sentence level often operate on the constituent words/tokens in a way that encodes the inductive bias of processing the input in a similar fashion to how humans do. However, there is no guarantee that the standard ordering of words is computationally efficient or optimal. To help mitigate this, we consider a dependency parse as a proxy for the inter-word dependencies in a sentence and simplify the sentence with respect to combinatorial objectives imposed on the… CONTINUE READING

    Figures, Tables, and Topics from this paper.


    Publications referenced by this paper.
    Do Grammars Minimize Dependency Length?
    • 95
    • PDF
    Large-scale evidence of dependency length minimization in 37 languages
    • 158
    • PDF
    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
    • 10,312
    • Highly Influential
    • PDF
    Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
    • 378
    • PDF
    Linguistic Knowledge and Transferability of Contextual Representations
    • 177
    • Highly Influential
    • PDF
    Sequence to Sequence Learning with Neural Networks
    • 10,547
    • Highly Influential
    • PDF
    Neural Machine Translation by Jointly Learning to Align and Translate
    • 12,937
    • PDF
    Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling
    • 330
    • PDF
    Attention is All you Need
    • 11,939
    • PDF
    Dependency distance: A new perspective on syntactic patterns in natural languages.
    • 78
    • PDF