A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks

@article{Sanh2019AHM,
  title={A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks},
  author={Victor Sanh and Thomas Wolf and Sebastian Ruder},
  journal={ArXiv},
  year={2019},
  volume={abs/1811.06031}
}
Much effort has been devoted to evaluate whether multi-task learning can be leveraged to learn rich representations that can be used in various Natural Language Processing (NLP) down-stream applications. [...] Key Method The model is trained in a hierarchical fashion to introduce an inductive bias by supervising a set of low level tasks at the bottom layers of the model and more complex tasks at the top layers of the model. This model achieves state-of-the-art results on a number of tasks, namely Named Entity…Expand
92 Citations

Paper Mentions

Multi-task Learning for Relation Extraction
Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning
  • 1
  • PDF
Multi-Task Learning for Coherence Modeling
  • 9
  • PDF
Hierarchical Multi-Task Natural Language Understanding for Cross-domain Conversational AI: HERMIT NLU
  • 9
  • Highly Influenced
  • PDF
Reevaluating Argument Component Extraction in Low Resource Settings
  • Highly Influenced
  • PDF
BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning
  • 65
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 46 REFERENCES
Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
  • 225
  • Highly Influential
  • PDF
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
  • 388
  • Highly Influential
  • PDF
Deep multi-task learning with low level tasks supervised at lower layers
  • 325
  • Highly Influential
  • PDF
Sluice networks: Learning what to share between loosely related tasks
  • 123
  • PDF
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
  • 1,189
  • PDF
Joint entity recognition and relation extraction as a multi-head selection problem
  • 88
  • Highly Influential
  • PDF
Named Entity Recognition with Bidirectional LSTM-CNNs
  • 1,020
  • Highly Influential
  • PDF
A Simple but Tough-to-Beat Baseline for Sentence Embeddings
  • 787
Latent Multi-Task Architecture Learning
  • 87
Multi-Task Cross-Lingual Sequence Tagging from Scratch
  • 173
  • PDF
...
1
2
3
4
5
...