Transfer Learning in Natural Language Processing

@inproceedings{Ruder2019TransferLI,
  title={Transfer Learning in Natural Language Processing},
  author={Sebastian Ruder and Matthew E. Peters and Swabha Swayamdipta and Thomas Wolf},
  booktitle={NAACL-HLT},
  year={2019}
}
The classic supervised machine learning paradigm is based on learning in isolation, a single predictive model for a task using a single dataset. This approach requires a large number of training examples and performs best for well-defined and narrow tasks. Transfer learning refers to a set of methods that extend this approach by leveraging data from additional domains or tasks to train a model with better generalization properties. Over the last two years, the field of Natural Language… Expand
94 Citations
Transformers : State-ofthe-art Natural Language Processing
  • 42
  • PDF
J an 2 02 0 O N THE COMPARABILITY OF PRE-TRAINED LANGUAGE MODELS
Low-Resource Adaptation of Neural NLP Models
  • PDF
Multi-pretraining for Large-scale Text Classification
  • Highly Influenced
  • PDF
Contextual Text Embeddings for Twi
  • PDF
Rethinking embedding coupling in pre-trained language models
  • 5
  • PDF
Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning
  • 1
  • PDF
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
  • 16
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 668 REFERENCES
Improving Language Understanding by Generative Pre-Training
  • 2,019
  • Highly Influential
  • PDF
Recurrent neural networks for language understanding
  • 259
  • PDF
Sequence to Sequence Learning with Neural Networks
  • 12,184
  • PDF
Matching Networks for One Shot Learning
  • 2,473
  • PDF
Multi-task Sequence to Sequence Learning
  • 596
  • PDF
Dissecting Contextual Word Embeddings: Architecture and Representation
  • 193
  • PDF
...
1
2
3
4
5
...