A Convolutional Neural Network for Modelling Sentences

@inproceedings{Kalchbrenner2014ACN,
  title={A Convolutional Neural Network for Modelling Sentences},
  author={Nal Kalchbrenner and Edward Grefenstette and Phil Blunsom},
  booktitle={ACL},
  year={2014}
}
The ability to accurately represent sentences is central to language understanding. [...] Key Method The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the…Expand
Tree-LSTM Guided Attention Pooling of DCNN for Semantic Sentence Modeling
TLDR
A novel structure to combine the strength of CNN, recurrent neural network and recursive neural networks for semantic modelling of sentences outperforms both Tree-LSTM and DCNN and achieves outstanding performance on sentiment classification task. Expand
Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents
TLDR
DSCNN hierarchically builds textual representations by processing pretrained word embeddings via Long ShortTerm Memory networks and subsequently extracting features with convolution operators, and does not rely on parsers and expensive phrase labeling, and thus is not restricted to sentencelevel tasks. Expand
A C-LSTM Neural Network for Text Classification
TLDR
C-LSTM is a novel and unified model for sentence representation and text classification that outperforms both CNN and LSTM and can achieve excellent performance on these tasks. Expand
Convolutional Neural Network for Universal Sentence Embeddings
TLDR
This paper proposes a simple CNN model that performs better than average models in transfer learning setting and exceeds the state of the art in supervised learning setting by initializing the parameters with the pre-trained sentence embeddings. Expand
An integrated neural model for sentence classification
TLDR
This work proposes a hybrid L-MCNN model to represent the semantics of sentences, and this model consists of two parts, bidirectional long short-term memory (LSTM) and multichannel convolutional neural networks (CNNs). Expand
Convolutional neural network for modelling sentences and its uses in sentiment analysis
The ability to accurately represent sentences is central to language understanding. Semantic modelling is at the core of many tasks involving a degree of natural language comprehension. These tasksExpand
Convolutional Recurrent Deep Learning Model for Sentence Classification
TLDR
This paper uses an unsupervised neural language model to train initial word embeddings that are further tuned by the authors' deep learning network, then, the pre-trained parameters of the network are used to initialize the model and a joint CNN and RNN framework is described to overcome the problem of loss of detailed, local information. Expand
Contextual-CNN: A Novel Architecture Capturing Unified Meaning for Sentence Classification
TLDR
A contextual CNN (C-CNN) is proposed for better text understanding by adding recurrent connection to the convolutional layer by helps CCNN units to be modulated over time with their neighboring units, thus the model integrates word meanings with surrounding information within the same layer. Expand
Character-Based Parsing with Convolutional Neural Network
TLDR
A novel convolutional neural network architecture with k-max pooling layer that is able to successfully recover the structure of Chinese sentences is described and archived competitive performance to existing benchmark parsers on the CTB-5 dataset without any task-specific feature engineering. Expand
Efficient Deep Learning Model for Text Classification Based on Recurrent and Convolutional Layers
TLDR
A neural language model that relies on Convolutional Neural Network and Bidirectional Recurrent Neural Network over pre-trained word vectors and utilizes bidirectional layers as a substitute of pooling layers in CNN to reduce the loss of detailed local information, and to capture long-term dependencies across input sequences. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 44 REFERENCES
A unified architecture for natural language processing: deep neural networks with multitask learning
We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semanticExpand
Recurrent Convolutional Neural Networks for Discourse Compositionality
TLDR
The discourse model coupled to the sentence model obtains state of the art performance on a dialogue act classification experiment and is able to capture both the sequentiality of sentences and the interaction between different speakers. Expand
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
TLDR
A Sentiment Treebank that includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality, and introduces the Recursive Neural Tensor Network. Expand
Context dependent recurrent neural network language model
TLDR
This paper improves recurrent neural network language models performance by providing a contextual real-valued input vector in association with each word to convey contextual information about the sentence being modeled by performing Latent Dirichlet Allocation using a block of preceding text. Expand
Grounded Compositional Semantics for Finding and Describing Images with Sentences
TLDR
The DT-RNN model, which uses dependency trees to embed sentences into a vector space in order to retrieve images that are described by those sentences, outperform other recursive and recurrent neural networks, kernelized CCA and a bag-of-words baseline on the tasks of finding an image that fits a sentence description and vice versa. Expand
Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
TLDR
A novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions that outperform other state-of-the-art approaches on commonly used datasets, without using any pre-defined sentiment lexica or polarity shifting rules. Expand
Recurrent Continuous Translation Models
We introduce a class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentencesExpand
Word Representations: A Simple and General Method for Semi-Supervised Learning
TLDR
This work evaluates Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeds of words on both NER and chunking, and finds that each of the three word representations improves the accuracy of these baselines. Expand
Inductive Learning in Symbolic Domains Using Structure-Driven Recurrent Neural Networks
TLDR
A connectionist architecture together with a novel supervised learning scheme which is capable of solving inductive inference tasks on complex symbolic structures of arbitrary size and first results from experiments with inductive learning tasks consisting in the classification of logical terms are given. Expand
LSTM recurrent networks learn simple context-free and context-sensitive languages
TLDR
Long short-term memory (LSTM) variants are also the first RNNs to learn a simple context-sensitive language, namely a(n)b( n)c(n). Expand
...
1
2
3
4
5
...