Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approach

@article{Jawanpuria2019LearningMW,
  title={Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approach},
  author={Pratik Jawanpuria and Arjun Balgovind and Anoop Kunchukuttan and Bamdev Mishra},
  journal={Transactions of the Association for Computational Linguistics},
  year={2019},
  volume={7},
  pages={107-120}
}
We propose a novel geometric approach for learning bilingual mappings given monolingual embeddings and a bilingual dictionary. Our approach decouples the source-to-target language transformation into (a) language-specific rotations on the original embeddings to align them in a common, latent space, and (b) a language-independent similarity metric in this common space to better model the similarity between the embeddings. Overall, we pose the bilingual mapping problem as a classification problem… Expand
Filtered Inner Product Projection for Multilingual Embedding Alignment
TLDR
This paper proposes a method, Filtered Inner Product Projection (FIPP), for mapping embeddings to a common representation space and evaluates FIPP in the context of bilingual dictionary induction and shows that the approach outperforms existing methods on the MUSE dataset for various language pairs. Expand
Filtered Inner Product Projection for Crosslingual Embedding Alignment
TLDR
Filtered Inner Product Projection outperforms existing methods on the XLING (5K) BLI dataset and theXLING (1K)BLI dataset, when using a self-learning approach, while also providing robust performance across downstream tasks. Expand
A Simple Approach to Learning Unsupervised Multilingual Embeddings
TLDR
A simple, two-stage framework in which the proposed approach obtains surprisingly good performance in various tasks such as bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing. Expand
Density Matching for Bilingual Word Embedding
TLDR
This paper proposes an approach that expresses the two monolingual embedding spaces as probability densities defined by a Gaussian mixture model, and matches the two densities using a method called normalizing flow, and argues that this formulation has several intuitively attractive properties. Expand
Geometry-aware domain adaptation for unsupervised alignment of word embeddings
TLDR
A novel manifold based geometric approach for learning unsupervised alignment of word embeddings between the source and the target languages by formulating the alignment learning problem as a domain adaptation problem over the manifold of doubly stochastic matrices. Expand
Evaluating Sub-word Embeddings in Cross-lingual Models
TLDR
The results indicate that cross-lingual representations for OOV words can indeed be formed from sub-word embeddings, including in the case of a truly low-resource morphologically-rich language. Expand
A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon Induction
TLDR
A novel graph-based paradigm to induce bilingual lexicons in a coarse-to-fine way that leverages clique-level information, which is richer and more accurate, but also effectively reduces the bad effect of the noise in the pre-trained embeddings. Expand
Unsupervised Hyperalignment for Multilingual Word Embeddings
TLDR
A novel formulation that ensures composable mappings, leading to better alignments is proposed, showing consistent improvement with indirect mappings while maintaining competitive performance on direct word translation. Expand
Joint Training for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora
TLDR
The results indicate that incorporating sub-word information indeed leads to improvements, and in the case of document classification, performance better than, or on par with, strong benchmark approaches. Expand
Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces
TLDR
A technique to quantitatively estimate the isometry between two embedding spaces and empirically show that this assumption weakens as the languages in question become increasingly etymologically distant is proposed. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 60 REFERENCES
Learning principled bilingual mappings of word embeddings while preserving monolingual invariance
TLDR
This paper proposes a framework that generalizes previous work, provides an efficient exact method to learn the optimal linear transformation and yields the best bilingual results in translation induction while preserving monolingual performance in an analogy task. Expand
Unsupervised Multilingual Word Embeddings
TLDR
This work proposes a fully unsupervised framework for learning MWEs that directly exploits the relations between all language pairs and substantially outperforms previous approaches in the experiments on multilingual word translation and cross-lingual word similarity. Expand
Improving Supervised Bilingual Mapping of Word Embeddings
TLDR
This work proposes to use a retrieval criterion instead of the square loss for learning the mapping of continuous word representations, and shows that this loss function leads to state-of-the-art results, with the biggest improvements observed for distant language pairs such as English-Chinese. Expand
Generalizing and Improving Bilingual Word Embedding Mappings with a Multi-Step Framework of Linear Transformations
TLDR
A multi-step framework of linear transformations that generalizes a substantial body of previous work is proposed that allows new insights into the behavior of existing methods, including the effectiveness of inverse regression, and design a novel variant that obtains the best published results in zero-shot bilingual lexicon extraction. Expand
Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion
TLDR
This paper proposes an unified formulation that directly optimizes a retrieval criterion in an end-to-end fashion for word translation, and shows that this approach outperforms the state of the art on word translation. Expand
An Autoencoder Approach to Learning Bilingual Word Representations
TLDR
This work explores the use of autoencoder-based methods for cross-language learning of vectorial word representations that are coherent between two languages, while not relying on word-level alignments, and achieves state-of-the-art performance. Expand
Translation Invariant Word Embeddings
TLDR
This work proposes a simple and scalable method that is inspired by the notion that the learned vector representations should be invariant to translation between languages, and shows empirically that this method outperforms prior work on multilingual tasks, matches the performance of Prior work on monolingual tasks, and scales linearly with the size of the input data. Expand
Improving Cross-Lingual Word Embeddings by Meeting in the Middle
TLDR
This work proposes to apply an additional transformation after the initial alignment step, which moves cross-lingual synonyms towards a middle point between them, and aims to obtain a better cross-lingsual integration of the vector spaces. Expand
BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
TLDR
It is shown that bilingual embeddings learned using the proposed BilBOWA model outperform state-of-the-art methods on a cross-lingual document classification task as well as a lexical translation task on WMT11 data. Expand
Unsupervised Alignment of Embeddings with Wasserstein Procrustes
TLDR
This paper proposes to use an alternative formulation, based on the joint estimation of an orthogonal matrix and a permutation matrix, for the task of aligning two sets of points in high dimension, which has many applications in natural language processing and computer vision. Expand
...
1
2
3
4
5
...