Syntax-Aware Multi-Sense Word Embeddings for Deep Compositional Models of Meaning

@inproceedings{Cheng2015SyntaxAwareMW,
  title={Syntax-Aware Multi-Sense Word Embeddings for Deep Compositional Models of Meaning},
  author={Jianpeng Cheng and Dimitri Kartsaklis},
  booktitle={EMNLP},
  year={2015}
}
Deep compositional models of meaning acting on distributional representations of words in order to produce vectors of larger text constituents are evolving to a popular area of NLP research. [...] Key Method Embeddings and composition layers are jointly learned against a generic objective that enhances the vectors with syntactic information from the surrounding context.Expand
Topic-Aware Deep Compositional Models for Sentence Classification
  • Rui Zhao, K. Mao
  • Computer Science
  • IEEE/ACM Transactions on Audio, Speech, and Language Processing
  • 2017
TLDR
Comprehensive experiments on five public sentence classification datasets are conducted and the results show that the proposed topic-aware deep compositional models produce competitive or better performance than other text representation learning methods. Expand
From lexical towards contextualized meaning representation
TLDR
This thesis proposes syntax-aware token embeddings (SATokE) that capture specific linguistic information, encoding the structure of the sentence from a dependency point of view in their representations and empirically demonstrates the superiority of the token representations compared to popular distributional representations of words and to other token embedDings proposed in the literature. Expand
Learning Context-Specific Word/Character Embeddings
TLDR
A neural network architecture to jointly learn word embeddings and context representations from large data sets is presented and the proposed model outperformed other competitors and is applicable to intrinsically "character-based" languages. Expand
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning
TLDR
This survey presents a comprehensive overview of the wide range of techniques in the two main branches of sense representation, i.e., unsupervised and knowledge-based and provides an analysis of four of its important aspects: interpretability, sense granularity, adaptability to different domains and compositionality. Expand
Learning to Represent Words in Context with Multilingual Supervision
TLDR
A neural network architecture based on bidirectional LSTMs to compute representations of words in the sentential contexts suitable for context-sensitive word representations, and obtains state-of-the-art results on all of these. Expand
Model-Free Context-Aware Word Composition
TLDR
This work proposes a model-free context-aware word composition model, which employs the latent semantic information as global context for learning representations, and attempts to resolve the word sense disambiguation and word composition in a unified framework. Expand
Constructing High Quality Sense-specific Corpus and Word Embedding via Unsupervised Elimination of Pseudo Multi-sense
TLDR
A novel framework for unsupervised corpus sense tagging is presented, and on the tasks of word similarity, word analogy as well as sentence understanding, the embeddings trained on sense-specific corpus obtain better results than the basic strategy which is applied in step (a). Expand
Understanding and Improving Multi-Sense Word Embeddings via Extended Robust Principal Component Analysis
TLDR
This paper addresses the pseudo multi-sense detection for word embeddings by dimensionality reduction of sense pairs with a novel principal analysis method, termed Ex-RPCA, designed to detect both pseudo multi senses and real multi senses. Expand
Compositional Distributional Models of Meaning
TLDR
This COLING 2016 tutorial aims at providing a concise introduction to this emerging field, presenting the different classes of CDMs and the various issues related to them in sufficient detail. Expand
Modelling Lexical Ambiguity with Density Matrices
TLDR
Three new neural models for learning density matrices from a corpus are presented and their ability to discriminate between word senses on a range of compositional datasets is tested, and the best model outperforms existing vector-based compositional models as well as strong sentence encoders. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 54 REFERENCES
Improving Word Representations via Global Context and Multiple Word Prototypes
TLDR
A new neural network architecture is presented which learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and accounts for homonymy and polysemy by learning multiple embedDings per word. Expand
Evaluating Neural Word Representations in Tensor-Based Compositional Settings
TLDR
In the more constrained tasks, co-occurrence vectors are competitive, although choice of compositional method is important; on the largerscale tasks, they are outperformed by neural word embeddings, which show robust, stable performance across the tasks. Expand
The Role of Syntax in Vector Space Models of Compositional Semantics
TLDR
This model leverages the CCG combinatory operators to guide a non-linear transformation of meaning within a sentence and is used to learn high dimensional embeddings for sentences and evaluate them in a range of tasks, demonstrating that the incorporation of syntax allows a concise model to learn representations that are both effective and general. Expand
Semantic Compositionality through Recursive Matrix-Vector Spaces
TLDR
A recursive neural network model that learns compositional vector representations for phrases and sentences of arbitrary syntactic type and length and can learn the meaning of operators in propositional logic and natural language is introduced. Expand
Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space
TLDR
An extension to the Skip-gram model that efficiently learns multiple embeddings per word type is presented, and its scalability is demonstrated by training with one machine on a corpus of nearly 1 billion tokens in less than 6 hours. Expand
Jointly Learning Word Representations and Composition Functions Using Predicate-Argument Structures
TLDR
A novel compositional language model that works on PredicateArgument Structures (PASs) that enables this model to capture longrange dependencies between words and to better handle constructs such as verbobject and subject-verb-object relations. Expand
Prior Disambiguation of Word Tensors for Constructing Sentence Vectors
TLDR
This paper proposes disambiguation algorithms for a number of tensor-based models, and tests the effectiveness of these algorithms on a variety of tasks, showing that disambIGuation can provide better compositional representation even for the case of tensors. Expand
A Comparison of Vector-based Representations for Semantic Composition
TLDR
Shallow approaches to modeling compositional meaning for phrases and sentences using distributional methods are found to be as good as more computationally intensive alternatives with regards to two particular tests: phrase similarity and paraphrase detection. Expand
Dynamic and Static Prototype Vectors for Semantic Composition
TLDR
The results show that selecting relevant senses of the constituent words leads to a better semantic composition of the compound, and dynamic prototypes perform better than static prototypes. Expand
GloVe: Global Vectors for Word Representation
TLDR
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure. Expand
...
1
2
3
4
5
...