• Corpus ID: 245704390

SMDT: Selective Memory-Augmented Neural Document Translation

  title={SMDT: Selective Memory-Augmented Neural Document Translation},
  author={Xu Zhang and Jian Yang and Haoyang Huang and Shuming Ma and Dongdong Zhang and Jinlong Li and Furu Wei},
Existing document-level neural machine translation (NMT) models have sufficiently explored different context settings to provide guidance for target generation. However, little attention is paid to inaugurate more diverse context for abundant context information. In this paper, we propose a Selective Memoryaugmented Neural Document Translation model (SMDT) to deal with documents containing large hypothesis space of the context. Specifically, we retrieve similar bilingual sentence pairs from the… 

Figures and Tables from this paper



Diverse Pretrained Context Encodings Improve Document Translation

Four general conclusions are supported: that using pre-trained context representations markedly improves sample efficiency, that adequate parallel data resources are crucial for learning to use document context, that jointly conditioning on multiple context representations outperforms any single representation, and that source context is more valuable for translation performance than target side context.

Selective Attention for Context-aware Neural Machine Translation

This work proposes a novel and scalable top-down approach to hierarchical attention for context-aware NMT which uses sparse attention to selectively focus on relevant sentences in the document context and then attends to key words in those sentences.

Document-Level Neural Machine Translation with Hierarchical Attention Networks

Experiments show that hierarchical attention significantly improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods, and that both the encoder and decoder benefit from context in complementary ways.

Hierarchical Modeling of Global Context for Document-Level Neural Machine Translation

This paper proposes a hierarchical model to learn the global context for document-level neural machine translation (NMT), which feedback the extracted global document context to each word in a top-down fashion to distinguish different translations of a word according to its specific surrounding context.

Improving the Transformer Translation Model with Document-Level Context

This work extends the Transformer model with a new context encoder to represent document-level context, which is then incorporated into the original encoder and decoder, and introduces a two-step training method to take full advantage of abundant sentence-level parallel corpora and limited document- level parallel Corpora.

Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation

This paper describes the Microsoft Translator submissions to the WMT19 news translation shared task for English-German and investigates document-level neural machine translation with deep transformer models, based on preliminary human evaluation results.

Efficient Context-Aware Neural Machine Translation with Layer-Wise Weighting and Input-Aware Gating

This paper proposes to make the most of layers pre-trained on sentence- level data in contextual representation learning, reusing representations from the sentence-level Transformer and significantly reducing the cost of incorporating contexts in translation.

Enhancing Context Modeling with a Query-Guided Capsule Network for Document-level Translation

A query-guided capsule networks to cluster context information into different perspectives from which the target translation may concern and can significantly outperform strong baselines on multiple data sets of different domains is proposed.

Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation

This paper proposes a surprisingly simple long-short term masking self-attention on top of the standard transformer to both effectively capture the long-range dependence and reduce the propagation of errors.

Towards Making the Most of Context in Neural Machine Translation

A new document-level NMT framework is proposed that deliberately models the local context of each sentence with the awareness of the global context of the document in both source and target languages.