Corpus ID: 215238601

Testing pre-trained Transformer models for Lithuanian news clustering

@inproceedings{Stankevivcius2020TestingPT,
  title={Testing pre-trained Transformer models for Lithuanian news clustering},
  author={L. Stankevivcius and Mantas Lukovsevivcius},
  year={2020}
}
A recent introduction of Transformer deep learning architecture made breakthroughs in various natural language processing tasks. However, non-English languages could not leverage such new opportunities with the English text pre-trained models. This changed with research focusing on multilingual models, where less-spoken languages are the main beneficiaries. We compare pre-trained multilingual BERT, XLM-R, and older learned text representation methods as encodings for the task of Lithuanian news… Expand

Figures and Tables from this paper

Russian News Clustering and Headline Selection Shared Task

References

SHOWING 1-10 OF 46 REFERENCES
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Sentiment Analysis of Lithuanian Texts Using Deep Learning Methods
Deep contextualized word representations
Unsupervised Cross-lingual Representation Learning at Scale
Language Models are Unsupervised Multitask Learners
Cross-lingual Language Model Pretraining
Attention is All you Need
...
1
2
3
4
5
...