Corpus ID: 211133077

Incorporating BERT into Neural Machine Translation

@article{Zhu2020IncorporatingBI,
  title={Incorporating BERT into Neural Machine Translation},
  author={Jinhua Zhu and Yingce Xia and Lijun Wu and Di He and Tao Qin and Wengang Zhou and Houqiang Li and Tie-Yan Liu},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.06823}
}
  • Jinhua Zhu, Yingce Xia, +5 authors Tie-Yan Liu
  • Published in ICLR 2020
  • Computer Science
  • The recently proposed BERT has shown great power on a variety of natural language understanding tasks, such as text classification, reading comprehension, etc. However, how to effectively apply BERT to neural machine translation (NMT) lacks enough exploration. While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning… CONTINUE READING

    Create an AI-powered research feed to stay up to date with new papers like this posted to ArXiv

    Citations

    Publications citing this paper.

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 42 REFERENCES

    Deep contextualized word representations

    VIEW 9 EXCERPTS
    HIGHLY INFLUENTIAL

    Cross-lingual Language Model Pretraining

    VIEW 12 EXCERPTS
    HIGHLY INFLUENTIAL

    Attention is All you Need

    VIEW 13 EXCERPTS
    HIGHLY INFLUENTIAL

    Improving Language Understanding by Generative Pre-Training

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Scaling Neural Machine Translation

    VIEW 2 EXCERPTS
    HIGHLY INFLUENTIAL