Corpus ID: 204509627

HuggingFace's Transformers: State-of-the-art Natural Language Processing

@article{Wolf2019HuggingFacesTS,
  title={HuggingFace's Transformers: State-of-the-art Natural Language Processing},
  author={Thomas Wolf and Lysandre Debut and Victor Sanh and Julien Chaumond and Clement Delangue and Anthony Moi and Pierric Cistac and Tim Rault and R'emi Louf and Morgan Funtowicz and Jamie Brew},
  journal={ArXiv},
  year={2019},
  volume={abs/1910.03771}
}
  • Thomas Wolf, Lysandre Debut, +8 authors Jamie Brew
  • Published 2019
  • Computer Science
  • ArXiv
  • Recent advances in modern Natural Language Processing (NLP) research have been dominated by the combination of Transfer Learning methods with large-scale language models, in particular based on the Transformer architecture. With them came a paradigm shift in NLP with the starting point for training a model on a downstream task moving from a blank specific model to a general-purpose pretrained architecture. Still, creating these general-purpose models remains an expensive and time-consuming… CONTINUE READING
    Few-shot Natural Language Generation for Task-Oriented Dialog
    • 16
    • PDF
    CamemBERT: a Tasty French Language Model
    • 65
    • PDF
    FlauBERT: Unsupervised Language Model Pre-training for French
    • 28
    • PDF
    Entity Matching with Transformer Architectures - A Step Forward in Data Integration
    • 8
    • PDF
    Poor Man's BERT: Smaller and Faster Transformer Models
    • 7
    • PDF
    Transformer-based End-to-End Question Generation
    • 2
    • PDF
    Byte Pair Encoding is Suboptimal for Language Model Pretraining
    • 6
    • Highly Influenced
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 60 REFERENCES
    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
    • 10,101
    • Highly Influential
    • PDF
    Language Models are Unsupervised Multitask Learners
    • 1,759
    • Highly Influential
    • PDF
    Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
    • 641
    • PDF
    Attention is All you Need
    • 11,795
    • PDF
    Improving Language Understanding by Generative Pre-Training
    • 1,422
    • Highly Influential
    • PDF
    GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
    • 864
    • Highly Influential
    • PDF
    Transfer Learning in Natural Language Processing
    • 50
    • PDF
    Universal Language Model Fine-tuning for Text Classification
    • 1,115
    • Highly Influential
    • PDF
    AllenNLP: A Deep Semantic Natural Language Processing Platform
    • 467
    • Highly Influential
    • PDF