Corpus ID: 225040574

mT5: A massively multilingual pre-trained text-to-text transformer

@article{Xue2020mT5AM,
  title={mT5: A massively multilingual pre-trained text-to-text transformer},
  author={Linting Xue and Noah Constant and A. Roberts and Mihir Kale and Rami Al-Rfou and Aditya Siddhant and A. Barua and Colin Raffel},
  journal={ArXiv},
  year={2020},
  volume={abs/2010.11934}
}

Linting Xue, Noah Constant, +5 authors Colin Raffel

Published 2020

Computer Science

ArXiv

The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. We describe the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks. All of the code and model… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed