Corpus ID: 7478738

Linguistic Regularities in Continuous Space Word Representations

@inproceedings{Mikolov2013LinguisticRI,
  title={Linguistic Regularities in Continuous Space Word Representations},
  author={Tomas Mikolov and Wen-tau Yih and Geoffrey Zweig},
  booktitle={NAACL},
  year={2013}
}
Continuous space language models have recently demonstrated outstanding results across a variety of tasks. [...] Key Result Remarkably, this method outperforms the best previous systems.Expand

Figures, Tables, and Topics from this paper

GloVe: Global Vectors for Word Representation
TLDR
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure. Expand
Morphological Smoothing and Extrapolation of Word Embeddings
TLDR
A latentvariable Gaussian graphical model is presented that allows us to extrapolate continuous representations for words not observed in the training corpus, as well as smoothing the representations provided for the observed words. Expand
Urdu Word Embeddings
TLDR
The skip-gram model is trained on more than 140 million Urdu words to create the first large-scale word embeddings for the Urdu language, which capture a high degree of syntactic and semantic similarity between words. Expand
Deriving Adjectival Scales from Continuous Space Word Representations
TLDR
This work pushes the interpretation of continuous space word representations further by demonstrating that vector offsets can be used to derive adjectival scales and evaluating the scales on the indirect answers to yes/no questions corpus. Expand
Better Word Representations with Recursive Neural Networks for Morphology
TLDR
This paper combines recursive neural networks, where each morpheme is a basic unit, with neural language models to consider contextual information in learning morphologicallyaware word representations and proposes a novel model capable of building representations for morphologically complex words from their morphemes. Expand
Semantic Regularities in Document Representations
TLDR
A new document analogy task is designed for testing the semantic regularities in document representations, and empirical evaluations over several state-of-the-art document representation models reveal that neural embedding based document representations work better on this analogy task than conventional methods. Expand
Pattern-based methods for Improved Lexical Semantics and Word Embeddings
Natural Language Processing (NLP) is a field of research that aims, on the one hand, to give computational answers to linguistic questions, and on the other hand, develop applications for languageExpand
Joint Word Representation Learning Using a Corpus and a Semantic Lexicon
TLDR
A joint word representation learning method that simultaneously predicts the co-occurrences of two words in a sentence subject to the relational constrains given by the semantic lexicon and statistically significantly outperforms previously proposed methods for incorporating semantic lexicons into word representations on several benchmark datasets. Expand
Discovering linguistic (ir)regularities in word embeddings through max-margin separating hyperplanes
TLDR
The model, SVMCos, is robust to a range of experimental choices when training word embeddings and finds that this representation of the relationship obtains the best results in dis-covering linguistic regularities. Expand
Sentence Analogies: Linguistic Regularities in Sentence Embeddings
TLDR
This paper investigates to what extent commonly used sentence vector representation spaces as well reflect certain kinds of regularities, and proposes a number of schemes to induce evaluation data, based on lexical analogy data as well as semantic relationships between sentences. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 25 REFERENCES
A Neural Probabilistic Language Model
TLDR
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences. Expand
Neural Probabilistic Language Models
TLDR
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences, and incorporates this new language model into a state-of-the-art speech recognizer of conversational speech. Expand
Efficient Estimation of Word Representations in Vector Space
TLDR
Two novel model architectures for computing continuous vector representations of words from very large data sets are proposed and it is shown that these vectors provide state-of-the-art performance on the authors' test set for measuring syntactic and semantic word similarities. Expand
Joint Learning of Words and Meaning Representations for Open-Text Semantic Parsing
TLDR
This work proposes a method that learns to assign MRs to a wide range of text thanks to a training scheme that combines learning from knowledge bases with learning from raw text. Expand
Continuous space language models
TLDR
Highly efficient learning algorithms are described that enable the use of training corpora of several hundred million words and it is shown that this approach can be incorporated into a large vocabulary continuous speech recognizer using a lattice rescoring framework at a very low additional processing time. Expand
Word Representations: A Simple and General Method for Semi-Supervised Learning
TLDR
This work evaluates Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeds of words on both NER and chunking, and finds that each of the three word representations improves the accuracy of these baselines. Expand
UTD: Determining Relational Similarity Using Lexical Patterns
TLDR
This approach achieved the best results on the SemEval 2012 Task 2, obtaining a Spearman correlation of 0.229 and an accuracy on reproducing human answers to MaxDiff questions of 39.4%. Expand
Distributed representations, simple recurrent networks, and grammatical structure
AbstractIn this paper three problems for a connectionist account of language are considered1.What is the nature of linguistic representations?2.How can complex structural relationships such asExpand
Discovering Binary Codes for Documents by Learning Deep Generative Models
TLDR
A deep generative model in which the lowest layer represents the word-count vector of a document and the top layer represents a learned binary code for that document is described, which allows more accurate and much faster retrieval than latent semantic analysis. Expand
Structured Output Layer neural network language model
TLDR
A new neural network language model (NNLM) based on word clustering to structure the output vocabulary: Structured Output Layer NNLM, able to handle vocabularies of arbitrary size, hence dispensing with the design of short-lists that are commonly used in NNLMs. Expand
...
1
2
3
...