# AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes

@inproceedings{Rothe2015AutoExtendEW,
title={AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes},
author={Sascha Rothe and Hinrich Sch{\"u}tze},
booktitle={ACL},
year={2015}
}
• Published in ACL 1 July 2015
• Computer Science
We present \textit{AutoExtend}, a system to learn embeddings for synsets and lexemes. It is flexible in that it can take any word embeddings as input and does not need an additional training corpus. The synset/lexeme embeddings obtained live in the same vector space as the word embeddings. A sparse tensor formalization guarantees efficiency and parallelizability. We use WordNet as a lexical resource, but AutoExtend can be easily applied to other resources like Freebase. AutoExtend achieves…
279 Citations

## Figures and Tables from this paper

Visualising WordNet Embeddings: some preliminary results
It is shown that the t-SNE co-ordinates can be used to reveal interesting semantic relations between word senses, and a new method is proposed that uses the simple x,y coordinates to compute semantic similarity.
Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy
• Computer Science
PACLIC
• 2021
Experimental results on multiple benchmarks for word sense disambiguation and sense discrimination tasks show that CDES can accurately learn sensespecific static embeddings reporting comparable performance to the current state-of-the-art sense embedDings.
Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy
Experimental results on multiple benchmarks for word sense disambiguation and sense discrimination tasks show that CDES can accurately learn sensespecific static embeddings reporting comparable performance to the current state-of-the-art sense embedDings.
Ontology-Aware Token Embeddings for Prepositional Phrase Attachment
• Computer Science
ACL
• 2017
Using context-sensitive embeddings in a model for predicting prepositional phrase (PP) attachments improves the accuracy of the PP attachment model by 5.4% absolute points, which amounts to a 34.
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training
• Computer Science
CoNLL
• 2017
This work proposes a new model which learns word and sense embeddings jointly and exploits large corpora and knowledge from semantic networks in order to produce a unified vector space of word and senses.
context2vec: Learning Generic Context Embedding with Bidirectional LSTM
• Computer Science
CoNLL
• 2016
This work presents a neural model for efficiently learning a generic context embedding function from large corpora, using bidirectional LSTM, and suggests they could be useful in a wide variety of NLP tasks.
HybridVec : Hybrid Distributional and Definitional Word Vectors
Results suggest that a combination of distributional vectors (GloVE embeddings) and definitional word vectors produced from an autoencoder provide an improvement for Neural Machine Translation and warrants further testing.
Inducing Conceptual Embedding Spaces from Wikipedia
• Computer Science
• 2017
This paper explores an automatic expansion of Google’s pretrained vectors using Wikipedia, adding millions of concepts and named entities in over 270 languages to reside in the same vector space, thus flexibly facilitating cross-lingual semantic applications.
CoKE : Word Sense Induction Using Contextualized Knowledge Embeddings
• Computer Science
AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering
• 2019
This work introduces a new approach that can induce polysemy to any pre-defined embedding space by jointly grounding contextualized sense representations learned from sense-tagged corpora and word embeddings to a knowledge base without the need for re-training.
Inducing Conceptual Embedding Spaces from Wikipedia
An automatic expansion of Google's pre-trained vectors using Wikipedia is explored, adding millions of concepts and named entities in over 270 languages to reside in the same vector space, thus flexibly facilitating cross-lingual semantic applications.

## References

SHOWING 1-10 OF 49 REFERENCES
Learning Sense-specific Word Embeddings By Exploiting Bilingual Resources
• Computer Science
COLING
• 2014
This paper addresses the problem of representing words with multiple and sense-specific embeddings, which are learned from bilingual parallel data by representing each word with a single embedding, which fails to capture polysemy.
Improving Word Representations via Global Context and Multiple Word Prototypes
• Computer Science
ACL
• 2012
A new neural network architecture is presented which learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and accounts for homonymy and polysemy by learning multiple embedDings per word.
Re-embedding words
• Computer Science
ACL
• 2013
This work proposes a method that takes as input an existing embedding, some labeled data, and produces an embedding in the same space, but with a better predictive performance in the supervised task.
Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space
• Computer Science
EMNLP
• 2014
An extension to the Skip-gram model that efficiently learns multiple embeddings per word type is presented, and its scalability is demonstrated by training with one machine on a corpus of nearly 1 billion tokens in less than 6 hours.
Knowledge-Powered Deep Learning for Word Embedding
• Computer Science
ECML/PKDD
• 2014
This study explores the capacity of leveraging morphological, syntactic, and semantic knowledge to achieve high-quality word embeddings, and explores these types of knowledge to define new basis for word representation, provide additional input information, and serve as auxiliary supervision in deep learning.
GloVe: Global Vectors for Word Representation
• Computer Science
EMNLP
• 2014
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
Improving Lexical Embeddings with Semantic Knowledge
• Computer Science
ACL
• 2014
This work proposes a new learning objective that incorporates both a neural language model objective (Mikolov et al., 2013) and prior knowledge from semantic resources to learn improved lexical semantic embeddings.
Better Word Representations with Recursive Neural Networks for Morphology
• Computer Science
CoNLL
• 2013
This paper combines recursive neural networks, where each morpheme is a basic unit, with neural language models to consider contextual information in learning morphologicallyaware word representations and proposes a novel model capable of building representations for morphologically complex words from their morphemes.
Learning Structured Embeddings of Knowledge Bases
• Computer Science
AAAI
• 2011
A learning process based on an innovative neural network architecture designed to embed any of these symbolic representations into a more flexible continuous vector space in which the original knowledge is kept and enhanced would allow data from any KB to be easily used in recent machine learning methods for prediction and information retrieval.
BabelNet: Building a Very Large Multilingual Semantic Network
• Computer Science
ACL
• 2010
A very large, wide-coverage multilingual semantic network that integrates lexicographic and encyclopedic knowledge from WordNet and Wikipedia and Machine Translation is also applied to enrich the resource with lexical information for all languages.