Corpus ID: 2407601

Distributed Representations of Sentences and Documents

@article{Le2014DistributedRO,
  title={Distributed Representations of Sentences and Documents},
  author={Quoc V. Le and Tomas Mikolov},
  journal={ArXiv},
  year={2014},
  volume={abs/1405.4053}
}
Many machine learning algorithms require the input to be represented as a fixed-length feature vector. [...] Key Method Our algorithm represents each document by a dense vector which is trained to predict words in the document. Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models. Empirical results show that Paragraph Vectors outperforms bag-of-words models as well as other techniques for text representations. Finally, we achieve new state-of-the-art results on…Expand
Supervised Paragraph Vector: Distributed Representations of Words, Documents and Class Labels
TLDR
A Supervised Paragraph Vector is proposed, a task-specific variant of Paragraph vector for situations where class labels exist, which uses class labels along with words and documents and obtains corresponding representations with respect to the particular classification task. Expand
Spherical Paragraph Model
TLDR
The Spherical Paragraph Model (SPM), a probabilistic generative model based on BoWE, is introduced for text representation and can fully leverage the rich semantics of words, the word co-occurrence information as well as the corpus-wide information to help the representation learning of texts. Expand
Learning Sentence Representation with Context
  • 2017
Vector representation of sentences is important for many text processing tasks that involve classifying, clustering, or ranking sentences. For solving these tasks, bag-of-word based representationExpand
Learning bag-of-embedded-words representations for textual information retrieval
TLDR
The proposed BoEW model provides a fast way to fine-tune the learned representation towards the information need of the user using relevance feedback techniques and a novel spherical entropy objective function is proposed to optimize the learning representation for retrieval using the cosine similarity metric. Expand
Deep Learning for Document Representation
TLDR
This chapter reviews the available document representation methods that include five deep learning-based approaches: Word2Vec, Doc2VEC, GloVe, LSTM, and CNN. Expand
Top2Vec: Distributed Representations of Topics
TLDR
This model does not require stop-word lists, stemming or lemmatization, and it automatically finds the number of topics, and the resulting topic vectors are jointly embedded with the document and word vectors with distance between them representing semantic similarity. Expand
Learning Sentence Vector Representations to Summarize
Summarization is a key task in natural language processing that has many practical use cases in the real world. One such use case is with regard to product or restaurant reviews, which often containExpand
Hierarchical Neural Language Models for Joint Representation of Streaming Documents and their Content
TLDR
Extensions to the proposed model, which can be applied to personalized recommendation and social relationship mining by adding further user layers to the hierarchy, thus learning user-specific vectors to represent individual preferences, are discussed. Expand
Text Categorization by Weighted Features
TLDR
This work proposes a novel text representation approach which considers word local and global importance via the BM25 weighing schema and uses word vectors pretrained from large text corpus to capture the latent semantic relatedness between words. Expand
Regularized and Retrofitted models for Learning Sentence Representation with Context
TLDR
This paper describes two types of contexts depending on their scope and utility and proposes two approaches to incorporate contextual information into content-based models, and demonstrates that the proposed models outshine existing models on three fundamental tasks. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 48 REFERENCES
Distributed Representations of Words and Phrases and their Compositionality
TLDR
This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling. Expand
A Neural Probabilistic Language Model
TLDR
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences. Expand
Neural Probabilistic Language Models
TLDR
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences, and incorporates this new language model into a state-of-the-art speech recognizer of conversational speech. Expand
A Neural Autoregressive Topic Model
TLDR
A new model for learning meaningful representations of text documents from an unlabeled collection of documents that takes inspiration from the conditional mean-field recursive equations of the Replicated Softmax to define a neural network architecture that estimates the probability of observing a new word in a given document given the previously observed words. Expand
Efficient Estimation of Word Representations in Vector Space
TLDR
Two novel model architectures for computing continuous vector representations of words from very large data sets are proposed and it is shown that these vectors provide state-of-the-art performance on the authors' test set for measuring syntactic and semantic word similarities. Expand
Linguistic Regularities in Continuous Space Word Representations
TLDR
The vector-space word representations that are implicitly learned by the input-layer weights are found to be surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset. Expand
Learning Word Vectors for Sentiment Analysis
TLDR
This work presents a model that uses a mix of unsupervised and supervised techniques to learn word vectors capturing semantic term--document information as well as rich sentiment content, and finds it out-performs several previously introduced methods for sentiment classification. Expand
Improving Word Representations via Global Context and Multiple Word Prototypes
TLDR
A new neural network architecture is presented which learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and accounts for homonymy and polysemy by learning multiple embedDings per word. Expand
Word Representations: A Simple and General Method for Semi-Supervised Learning
TLDR
This work evaluates Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeds of words on both NER and chunking, and finds that each of the three word representations improves the accuracy of these baselines. Expand
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
TLDR
A Sentiment Treebank that includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality, and introduces the Recursive Neural Tensor Network. Expand
...
1
2
3
4
5
...