A Multitask Objective to Inject Lexical Contrast into Distributional Semantics

@inproceedings{Pham2015AMO,
  title={A Multitask Objective to Inject Lexical Contrast into Distributional Semantics},
  author={Nghia The Pham and Angeliki Lazaridou and Marco Baroni},
  booktitle={ACL},
  year={2015}
}
Distributional semantic models have trouble distinguishing strongly contrasting words (such as antonyms) from highly compatible ones (such as synonyms), because both kinds tend to occur in similar contexts in corpora. We introduce the multitask Lexical Contrast Model (mLCM), an extension of the effective Skip-gram method that optimizes semantic vectors on the joint tasks of predicting corpus contexts and making the representations of WordNet synonyms closer than that of matching WordNet… Expand
Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinction
TLDR
A novel vector representation that integrates lexical contrast into distributional vectors and strengthens the most salient features for determining degrees of word similarity and integrated into the objective function of a skip-gram model is proposed. Expand
Antonym-Synonym Classification Based on New Sub-space Embeddings
TLDR
Experimental results show that the proposed model outperforms existing research on antonym synonym distinction in both speed and performance. Expand
Injecting Lexical Contrast into Word Vectors by Guiding Vector Space Specialisation
TLDR
It is shown that a careful selection of the external constraints can steer and improve the specialisation of word vector space specialisation models. Expand
A Neural Network Model for Efficient Antonymy-Synonymy Classification by Exploiting Co-occurrence Contexts and Word-Structure Patterns
TLDR
A deep neural network model (DVASNet) is proposed that can utilize not only embedding representations of words but also co-occurrence contexts and specific patterns of Vietnamese word structure and achieved significant improvements in comparison with a number of state-of-the-art methods. Expand
Distinguishing antonymy, synonymy and hypernymy with distributional and distributed vector representations and neural networks
TLDR
A novel weighted feature for constructing word vectors by relying on distributional lexical contrast, a feature capable of differentiating between antonymy and synonymy, and a novel neural model to learn hierarchical embeddings for hypernymy detection and directionality are proposed. Expand
Extending WordNet with Fine-Grained Collocational Information via Supervised Distributional Learning
TLDR
ColWordNet (CWN) is presented, an extended WordNet version with fine-grained collocational information, automatically introduced thanks to a method exploiting linear relations between analogous sense-level embeddings spaces. Expand
Simplified Natural Language Processing and Discrimination in Antonyms-Synonyms Word Structure in a Neural Network Approach to Exploit Lexico-Semantic Pattern
To get high efficiency or high yielding in NLP system the vital challenge is to distinguish between antonym and synonym. By using lexico-syntactic patterns in pattern-based models, which areExpand
Distinguishing between paradigmatic semantic relations across word classes: human ratings and distributional similarity
TLDR
The fact that the most salient vector spaces and their success vary across word classes and paradigmatic relations suggests that combining feature types for relation distinction is better than applying them in isolation, and even a simple classification model is able to manage this task. Expand
SemEval-2017 Task 2: Multilingual and Cross-lingual Semantic Word Similarity
TLDR
Results show that systems that combine statistical knowledge from text corpora, in the form of word embeddings, and external knowledge from lexical resources are best performers in both subtasks. Expand
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training
TLDR
This work proposes a new model which learns word and sense embeddings jointly and exploits large corpora and knowledge from semantic networks in order to produce a unified vector space of word and senses. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 27 REFERENCES
Taking Antonymy Mask off in Vector Space
TLDR
APAnt is introduced, a new AveragePrecision-based measure for the unsupervised discrimination of antonymy from synonymy using Distributional Semantic Models (DSMs) using Average Precision to estimate the extent and salience of the intersection among the most descriptive contexts of two target words. Expand
Word Embedding-based Antonym Detection using Thesauri and Distributional Information
TLDR
This paper proposes a novel approach to train word embeddings to capture antonyms by utilizing supervised synonym and antonym information from thesauri, as well as distributional information from large-scale unlabelled text data. Expand
A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations
TLDR
A supervised corpus-based machine learning algorithm is introduced for classifying analogous word pairs and it is shown that it can solve multiple-choice SAT analogy questions, TOEFL synonyms questions, ESL synonym-antonym questions, and similar-associated-both questions from cognitive psychology. Expand
Pattern-Based Distinction of Paradigmatic Relations for German Nouns, Verbs, Adjectives
TLDR
A simple vector space model relying on lexico-syntactic patterns to distinguish between the paradigmatic Relations synonymy, antonymy and hypernymy shows that antonym/synonym distinction is performed significantly better than synonym/hypernym distinction, and that paradigmatic relations between verbs are more difficult to predict than paradigm relations between nouns or adjectives. Expand
Multimodal Distributional Semantics
TLDR
This work proposes a flexible architecture to integrate text- and image-based distributional information, and shows in a set of empirical tests that the integrated model is superior to the purely text-based approach, and it provides somewhat complementary semantic information with respect to the latter. Expand
Computing Lexical Contrast
TLDR
An automatic and empirical measure of lexical contrast that relies on the contrast hypothesis, corpus statistics, and the structure of a Roget-like thesaurus is presented, which obtains high precision and large coverage, outperforming existing methods. Expand
SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation
TLDR
SimLex-999 is presented, a gold standard resource for evaluating distributional semantic models that improves on existing resources in several important ways, and explicitly quantifies similarity rather than association or relatedness so that pairs of entities that are associated but not actually similar have a low rating. Expand
So similar and yet incompatible: Toward the automated identification of semantically compatible words
TLDR
A publicly available data-set of human compatibility ratings is presented, and a neural-network model that takes distributional embeddings of words as input and learns alternativeembeddings that perform the compatibility detection task quite well is presented. Expand
Deriving Adjectival Scales from Continuous Space Word Representations
TLDR
This work pushes the interpretation of continuous space word representations further by demonstrating that vector offsets can be used to derive adjectival scales and evaluating the scales on the indirect answers to yes/no questions corpus. Expand
Incorporating Both Distributional and Relational Semantics in Word Representations
TLDR
Preliminary results on knowledge base completion, analogy tests, and parsing show that word representations trained on both distributional and relational semantics can give improvements in some cases. Expand
...
1
2
3
...