Corpus ID: 211133077

Incorporating BERT into Neural Machine Translation

@article{Zhu2020IncorporatingBI,
  title={Incorporating BERT into Neural Machine Translation},
  author={Jinhua Zhu and Yingce Xia and Lijun Wu and Di He and Tao Qin and W. Zhou and H. Li and T. Liu},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.06823}
}
The recently proposed BERT has shown great power on a variety of natural language understanding tasks, such as text classification, reading comprehension, etc. However, how to effectively apply BERT to neural machine translation (NMT) lacks enough exploration. While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning… Expand
68 Citations
Dynamic Attention Aggregation with BERT for Neural Machine Translation
  • PDF
Document-Level Neural Machine Translation Using BERT as Context Encoder
  • Highly Influenced
  • PDF
On Long-Tailed Phenomena in Neural Machine Translation
  • 1
  • Highly Influenced
  • PDF
BERT Enhanced Neural Machine Translation and Sequence Tagging Model for Chinese Grammatical Error Diagnosis
  • Highly Influenced
  • PDF
BERT-JAM: Boosting BERT-Enhanced Neural Machine Translation with Joint Attention
  • Highly Influenced
  • PDF
Diving Deep into Context-Aware Neural Machine Translation
  • 1
  • PDF
Better Neural Machine Translation by Extracting Linguistic Information from BERT
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 43 REFERENCES
Towards Making the Most of BERT in Neural Machine Translation
  • 35
  • PDF
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  • 16,470
  • Highly Influential
  • PDF
MASS: Masked Sequence to Sequence Pre-training for Language Generation
  • 330
  • PDF
Phrase-Based & Neural Unsupervised Machine Translation
  • 399
  • PDF
Improving Language Understanding by Generative Pre-Training
  • 1,969
  • Highly Influential
  • PDF
Improving Neural Machine Translation Models with Monolingual Data
  • 1,204
  • Highly Influential
  • PDF
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
  • 3,468
  • PDF
...
1
2
3
4
5
...