No Training Required: Exploring Random Encoders for Sentence Classification
@article{Wieting2019NoTR, title={No Training Required: Exploring Random Encoders for Sentence Classification}, author={J. Wieting and Douwe Kiela}, journal={ArXiv}, year={2019}, volume={abs/1901.10444} }
We explore various methods for computing sentence representations from pre-trained word embeddings without any training, i.e., using nothing but random parameterizations. Our aim is to put sentence embeddings on more solid footing by 1) looking at how much modern sentence embeddings gain over random methods---as it turns out, surprisingly little; and by 2) providing the field with more appropriate baselines going forward---which are, as it turns out, quite strong. We also make important… CONTINUE READING
Supplemental Code
Github Repo
Via Papers with Code
Exploring Random Encoders for Sentence Classification
Figures, Tables, and Topics from this paper
Paper Mentions
60 Citations
How to Probe Sentence Embeddings in Low-Resource Languages: On Structural Design Choices for Probing Task Evaluation
- Computer Science
- CoNLL
- 2020
- 3
- PDF
Quantifying the Contextualization of Word Representations with Semantic Class Probing
- Computer Science
- EMNLP
- 2020
- 2
- PDF
On the impressive performance of randomly weighted encoders in summarization tasks
- Computer Science
- ACL 2019
- 2019
- 1
- PDF
Learning Compressed Sentence Representations for On-Device Text Processing
- Computer Science
- ACL
- 2019
- 9
- PDF
References
SHOWING 1-10 OF 62 REFERENCES
What you can cram into a single vector: Probing sentence embeddings for linguistic properties
- Computer Science
- ACL
- 2018
- 336
- Highly Influential
- PDF
Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
- Computer Science
- NAACL-HLT
- 2018
- 345
- PDF
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
- Computer Science
- ICLR
- 2017
- 281
- PDF
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
- Computer Science
- EMNLP
- 2017
- 1,135
- PDF
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
- Computer Science
- ACL
- 2018
- 169
- PDF
Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations
- Computer Science, Mathematics
- ACL
- 2018
- 136
- PDF