Three Strategies to Improve One-to-Many Multilingual Translation

@inproceedings{Wang2018ThreeST,
  title={Three Strategies to Improve One-to-Many Multilingual Translation},
  author={Yining Wang and Jiajun Zhang and Feifei Zhai and Jingfang Xu and Chengqing Zong},
  booktitle={EMNLP},
  year={2018}
}
Due to the benefits of model compactness, multilingual translation (including many-toone, many-to-many and one-to-many) based on a universal encoder-decoder architecture attracts more and more attention. [...] Key Method Within the architecture of one decoder for all target languages, we first exploit the use of unique initial states for different target languages. Then, we employ language-dependent positional embeddings.Expand
A Compact and Language-Sensitive Multilingual Translation Method
TLDR
To maximize parameter sharing, this work presents a universal representor to replace both encoder and decoder models, and introduces language-sensitive embedding, attention, and discriminator with the ability to enhance model performance. Expand
Improving Multilingual Translation by Representation and Gradient Regularization
Multilingual Neural Machine Translation (NMT) enables one model to serve all translation directions, including ones that are unseen during training, i.e. zero-shot translation. Despite beingExpand
Meet Changes with Constancy: Learning Invariance in Multi-Source Translation
TLDR
This paper proposes a source invariance network to learn the invariant information of parallel sources and demonstrates that the proposed approach not only achieves clear gains in translation quality but also captures implicit invariance between different sources. Expand
An Evaluation of Language-Agnostic Inner-Attention-Based Representations in Machine Translation
TLDR
It is demonstrated that the performance in translation does correlate with trainable downstream tasks, and it is shown that larger intermediate layers not only improve translation quality, especially for long sentences, but also push the accuracy of trainable classification tasks. Expand
A Systematic Study of Inner-Attention-Based Sentence Representations in Multilingual Neural Machine Translation
TLDR
This article explores a multilingual translation model capable of producing fixed-size sentence representations by incorporating an intermediate crosslingual shared layer, which is referred to as attention bridge, and shows that larger intermediate layers not only improve translation quality, especially for long sentences, but also push the accuracy of trainable classification tasks. Expand
A Comprehensive Survey of Multilingual Neural Machine Translation
TLDR
An in-depth survey of existing literature on multilingual neural machine translation is presented, which categorizes various approaches based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, core-issues and challenges. Expand
A Survey of Multilingual Neural Machine Translation
TLDR
An in-depth survey of existing literature on multilingual neural machine translation (MNMT) is presented and various approaches are categorized based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, core-issues, and challenges. Expand
Beyond English-Centric Multilingual Machine Translation
TLDR
This work creates a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages and explores how to effectively increase model capacity through a combination of dense scaling and language-specific sparse parameters to create high quality models. Expand
Improving Multilingual Neural Machine Translation For Low-Resource Languages: French, English - Vietnamese
TLDR
This paper proposes two simple strategies to address the rare word issue in multilingual MT systems for two low-resource language pairs: French-Vietnamese and English-vietnamese. Expand
Massively Multilingual Neural Machine Translation
TLDR
It is shown that massively multilingual many-to-many models are effective in low resource settings, outperforming the previous state-of-the-art while supporting up to 59 languages in 116 translation directions in a single model. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 30 REFERENCES
Multilingual Neural Machine Translation with Task-Specific Attention
TLDR
This work proposes task-specific attention models, a simple but effective technique for improving the quality of sequence-to-sequence neural multilingual translation that seeks to retain as much of the parameter sharing generalization of NMT models as possible, while still allowing for language-specific specialization of the attention model to a particular language-pair or task. Expand
Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
In this paper, we present our first attempts in building a multilingual Neural Machine Translation framework under a unified approach. We are then able to employ attention-based NMT for many-to-manyExpand
A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation
TLDR
This work provides a quantitative and comparative analysis of the translations produced by bilingual, multilingual and zero- shot systems; investigates the translation quality of two of the currently dominant neural architectures in MT, which are the Recurrent and the Transformer ones; and quantitatively explores how the closeness between languages influences the zero-shot translation. Expand
Improving Neural Machine Translation Models with Monolingual Data
TLDR
This work pairs monolingual training data with an automatic back-translation, and can treat it as additional parallel training data, and obtains substantial improvements on the WMT 15 task English German, and for the low-resourced IWSLT 14 task Turkish->English. Expand
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parametersExpand
Multi-task Sequence to Sequence Learning
TLDR
The results show that training on a small amount of parsing and image caption data can improve the translation quality between English and German by up to 1.5 BLEU points over strong single-task baselines on the WMT benchmarks, and reveal interesting properties of the two unsupervised learning objectives, autoencoder and skip-thought, in the MTL context. Expand
Exploiting Source-side Monolingual Data in Neural Machine Translation
TLDR
Two approaches to make full use of the sourceside monolingual data in NMT are proposed using the self-learning algorithm to generate the synthetic large-scale parallel data for NMT training and the multi-task learning framework using two NMTs to predict the translation and the reordered source-side monolingUAL sentences simultaneously. Expand
Attention is All you Need
TLDR
A new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely is proposed, which generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data. Expand
Transfer Learning for Low-Resource Neural Machine Translation
TLDR
A transfer learning method is presented that significantly improves Bleu scores across a range of low-resource languages by first train a high-resource language pair, then transfer some of the learned parameters to the low- resource pair to initialize and constrain training. Expand
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
TLDR
This work proposes a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages using a shared wordpiece vocabulary, and introduces an artificial token at the beginning of the input sentence to specify the required target language. Expand
...
1
2
3
...