Corpus ID: 12268909

Embedding Word Similarity with Neural Machine Translation

@article{Hill2015EmbeddingWS,
  title={Embedding Word Similarity with Neural Machine Translation},
  author={Felix Hill and Kyunghyun Cho and S{\'e}bastien Jean and Coline Devin and Yoshua Bengio},
  journal={CoRR},
  year={2015},
  volume={abs/1412.6448}
}
Neural language models learn word representations, or embeddings, that capture rich linguistic and conceptual information. [...] Key Method We further show that these effects hold when translating from both English to French and English to German, and argue that the desirable properties of translation embeddings should emerge largely independently of the source and target languages.Expand
Word and Phrase Translation with word2vec
TLDR
The scope of previous attempts of bilingual translation to four languages is expanded, showing how to process the source data, train a neural network to learn the high-dimensional embeddings for individual languages and expands the framework for testing their quality beyond the English language. Expand
Learning Lexical Embeddings with Syntactic and Lexicographic Knowledge
TLDR
This work hypothesizes that lexical embedding learning can further benefit from factorizing syntactic relations into individual relations for structured syntactic information and defining relatedness using lexicographic knowledge, and shows that implementation of these ideas brings notable improvement in lexical similarity tasks. Expand
Knowledge-Enhanced Ensemble Learning for Word Embeddings
TLDR
A knowledge-enhanced ensemble method to combine both knowledge graphs and pre-trained word embedding models is proposed, which interprets relations in knowledge graphs as linear translation from one word to another. Expand
Learning Word Meta-Embeddings by Autoencoding
TLDR
This work proposes three flavours of autoencoded meta- embeddings motivated by different requirements that must be satisfied by a meta-embedding, and shows that the proposed autoencodes outperform the existing state-of-the-art meta- embeddeds in multiple tasks. Expand
Deep Generative Model for Joint Alignment and Word Representation
This work exploits translation data as a source of semantically relevant learning signal for models of word representation. In particular, we exploit equivalence through translation as a form ofExpand
Think Globally, Embed Locally - Locally Linear Meta-embedding of Words
TLDR
Experimental results on semantic similarity, word analogy, relation classification, and short-text classification tasks show that the proposed unsupervised locally linear meta-embedding learning method to significantly outperform prior methods in several benchmark datasets, establishing a new state of the art for meta- embeddings. Expand
Word Embedding for Understanding Natural Language: A Survey
TLDR
This survey introduces the motivation and background of word embedding, the methods of text representation as preliminaries, as well as some existingword embedding approaches such as Neural Network Language Model and Sparse Coding Approach, along with their evaluation metrics. Expand
Learning to Generate Word Representations using Subword Information
TLDR
Experimental results show clearly that the proposed model significantly outperforms strong baseline models that regard words or their subwords as atomic units and as much as 18.5% improvement on average in perplexity for morphologically rich languages compared to strong baselines in the language modeling task. Expand
Adaptation of neural machine translation systems to domain-specific vocabulary
TLDR
The goal of this work is to solve the vocabulary size limitation that current NMT systems have and also automate the translation enforcement for some domain-specific words, which means that some words will always need to be translated to other specific words as synonyms do not have the same exact meaning. Expand
Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext
We consider the problem of learning general-purpose, paraphrastic sentence embeddings in the setting of Wieting et al. (2016b). We use neural machine translation to generate sentential paraphrasesExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 42 REFERENCES
Multilingual Distributed Representations without Word Alignment
TLDR
This work proposes a method for learning distributed representations in a multilingual setup and shows that these representations are semantically informative and apply them to a cross-lingual document classification task where they outperform the previous state of the art. Expand
An Autoencoder Approach to Learning Bilingual Word Representations
TLDR
This work explores the use of autoencoder-based methods for cross-language learning of vectorial word representations that are coherent between two languages, while not relying on word-level alignments, and achieves state-of-the-art performance. Expand
Neural Machine Translation by Jointly Learning to Align and Translate
TLDR
It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. Expand
Recurrent Continuous Translation Models
We introduce a class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentencesExpand
On Using Very Large Target Vocabulary for Neural Machine Translation
TLDR
It is shown that decoding can be efficiently done even with the model having a very large target vocabulary by selecting only a small subset of the whole target vocabulary. Expand
Addressing the Rare Word Problem in Neural Machine Translation
TLDR
This paper proposes and implements an effective technique to address the problem of end-to-end neural machine translation's inability to correctly translate very rare words, and is the first to surpass the best result achieved on a WMT’14 contest task. Expand
Exploiting Similarities among Languages for Machine Translation
TLDR
This method can translate missing word and phrase entries by learning language structures based on large monolingual data and mapping between languages from small bilingual data and uses distributed representation of words and learns a linear mapping between vector spaces of languages. Expand
GloVe: Global Vectors for Word Representation
TLDR
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure. Expand
Multimodal Distributional Semantics
TLDR
This work proposes a flexible architecture to integrate text- and image-based distributional information, and shows in a set of empirical tests that the integrated model is superior to the purely text-based approach, and it provides somewhat complementary semantic information with respect to the latter. Expand
Improving Vector Space Word Representations Using Multilingual Correlation
TLDR
This paper argues that lexico-semantic content should additionally be invariant across languages and proposes a simple technique based on canonical correlation analysis (CCA) for incorporating multilingual evidence into vectors generated monolingually. Expand
...
1
2
3
4
5
...