Corpus ID: 59336240

No Training Required: Exploring Random Encoders for Sentence Classification

  title={No Training Required: Exploring Random Encoders for Sentence Classification},
  author={J. Wieting and Douwe Kiela},
  • J. Wieting, Douwe Kiela
  • Published 2019
  • Computer Science
  • ArXiv
  • We explore various methods for computing sentence representations from pre-trained word embeddings without any training, i.e., using nothing but random parameterizations. Our aim is to put sentence embeddings on more solid footing by 1) looking at how much modern sentence embeddings gain over random methods---as it turns out, surprisingly little; and by 2) providing the field with more appropriate baselines going forward---which are, as it turns out, quite strong. We also make important… CONTINUE READING
    60 Citations

    Figures, Tables, and Topics from this paper

    Explore Further: Topics Discussed in This Paper

    Pitfalls in the Evaluation of Sentence Embeddings
    • 6
    • PDF
    Neural Language Priors
    • 2
    • PDF
    What do you mean, BERT?
    • PDF
    On the impressive performance of randomly weighted encoders in summarization tasks
    • 1
    • PDF
    Learning Compressed Sentence Representations for On-Device Text Processing
    • 9
    • PDF
    COSTRA 1.0: A Dataset of Complex Sentence Transformations
    • 3
    • PDF


    What you can cram into a single vector: Probing sentence embeddings for linguistic properties
    • 336
    • Highly Influential
    • PDF
    A Simple but Tough-to-Beat Baseline for Sentence Embeddings
    • 741
    Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
    • 345
    • PDF
    Skip-Thought Vectors
    • 1,629
    • Highly Influential
    • PDF
    Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
    • 281
    • PDF
    Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
    • 1,135
    • PDF
    Towards Universal Paraphrastic Sentence Embeddings
    • 418
    • PDF
    Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
    • 169
    • PDF
    Universal Sentence Encoder
    • 630
    • PDF
    Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations
    • 136
    • PDF