• Corpus ID: 64908139

A Simple but Tough-to-Beat Baseline for Sentence Embeddings

@inproceedings{Arora2017ASB,
  title={A Simple but Tough-to-Beat Baseline for Sentence Embeddings},
  author={Sanjeev Arora and Yingyu Liang and Tengyu Ma},
  booktitle={ICLR},
  year={2017}
}
Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline
TLDR
This paper first shows that word vector length has a confounding effect on the probability of a sentence being generated in Arora et al.
Sentence Embeddings using Supervised Contrastive Learning
TLDR
This paper fine-tunes pretrained BERT on SNLI data, incorporating both supervised crossentropy loss and supervised contrastive loss, and proposes a new method to build sentence embeddings by doing supervised Contrastive learning.
Sequential Sentence Embeddings for Semantic Similarity
TLDR
This work proposes a novel approach to compute sentence embeddings for semantic similarity that exploits a linear autoencoder for sequences and provides a grounded approach to identify and subtract common discourse from a sentence and its embedding, to remove associated uninformative features.
MappSent: a Textual Mapping Approach for Question-to-Question Similarity
TLDR
MappSent is introduced, a novel approach for textual similarity based on a linear sentence embedding representation that maps sentences in a joint-subspace where similar sets of sentences are pushed closer and achieves competitive results and outperforms in most cases state-of-art methods.
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors
TLDR
A la carte embedding is introduced, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings.
Meta-Embeddings for Natural Language Inference and Semantic Similarity tasks
TLDR
Meta Embedding derived from few State-of-the-Art (SOTA) models are proposed to efficiently tackle mainstream NLP tasks like classification, semantic relatedness, and text similarity by showing us that meta-embeddings can be used for several NLP task by harnessing the power of several individual representations.
Evaluation of sentence embeddings in downstream and linguistic probing tasks
TLDR
It is shown that a simple approach using bag-of-words with a recently introduced language model for deep context-dependent word embeddings proved to yield better results in many tasks when compared to sentence encoders trained on entailment datasets.
On the Sentence Embeddings from Pre-trained Language Models
TLDR
This paper proposes to transform the anisotropic sentence embedding distribution to a smooth and isotropic Gaussian distribution through normalizing flows that are learned with an unsupervised objective and achieves significant performance gains over the state-of-the-art sentence embeddings on a variety of semantic textual similarity tasks.
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
TLDR
This paper conducts a point-by-point comparative study between Simple Word-Embedding-based Models (SWEMs), consisting of parameter-free pooling operations, relative to word-embedding-based RNN/CNN models, and proposes two additional pooling strategies over learned word embeddings: a max-pooling operation for improved interpretability and a hierarchical pooling operation, which preserves spatial information within text sequences.
An Empirical Study on Post-processing Methods for Word Embeddings
TLDR
By optimising an objective in the semi-Riemannian manifold with Centralised Kernel Alignment, this work is able to search for the optimal shrinkage parameter, and provide a post-processing method to smooth the spectrum of learnt word vectors which yields improved performance on downstream tasks.
...
1
2
3
4
5
...