Understanding Convolutional Neural Networks for Text Classification

@inproceedings{Jacovi2018UnderstandingCN,
  title={Understanding Convolutional Neural Networks for Text Classification},
  author={Alon Jacovi and Oren Sar Shalom and Y. Goldberg},
  booktitle={BlackboxNLP@EMNLP},
  year={2018}
}
We present an analysis into the inner workings of Convolutional Neural Networks (CNNs) for processing text. CNNs used for computer vision can be interpreted by projecting filters into image space, but for discrete sequence inputs CNNs remain a mystery. We aim to understand the method by which the networks process and classify text. We examine common hypotheses to this problem: that filters, accompanied by global max-pooling, serve as ngram detectors. We show that filters may capture several… Expand
On the Interpretation of Convolutional Neural Networks for Text Classification
TLDR
A mathematical decomposition is presented to translate the output of CNN for text classification into an ngram- level score matrix and a word-level score matrix, revealing how various parts of input sentences contribute to the final prediction quantitatively. Expand
Interpretable Text Classification Using CNN and Max-pooling
TLDR
This work studies the interpretability of a variant of the typical text classification model which is based on convolutional operation and max-pooling layer and evaluates the performance of the model on several classification tasks and justifies the interpretable performance with some case studies. Expand
Classifying Short Text for the Harmonized System with Convolutional Neural Networks
  • 2019
Classifying goods in the hierarchical Harmonized System is a challenging problem for both humans and machines. To combat this, we propose a Convolutional Neural Network (CNN) architecture to labelExpand
Token-wise sentiment decomposition for ConvNet: Visualizing a sentiment classifier
TLDR
This work presents a visualization technique that can be used to understand the inner workings of text-based CNN models and shows how this method can be use to generate adversarial examples and learn the shortcomings of the training data. Expand
Evaluating Recurrent Neural Network Explanations
TLDR
Using the method that performed best in the authors' experiments, it is shown how specific linguistic phenomena such as the negation in sentiment analysis reflect in terms of relevance patterns, and how the relevance visualization can help to understand the misclassification of individual samples. Expand
Combining word embeddings and convolutional neural networks to detect duplicated questions
TLDR
This work proposes a simple approach to identifying semantically similar questions by combining the strengths of word embeddings and Convolutional Neural Networks (CNNs), and demonstrates how the cosine similarity metric can be used to effectively compare feature vectors. Expand
ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models
TLDR
It is shown that simple models based on CNN or LSTM as well as complex models like BERT are questionable in terms of their syntactic and semantic understanding. Expand
Simplifying the explanation of deep neural networks with sufficient and necessary feature-sets: case of text classification
TLDR
A method to simplify the prediction explanation of One-Dimensional (1D) Convolutional Neural Networks (CNN) by identifying sufficient and necessary features-sets and an adaptation of Layer-wise Relevance Propagation for 1D-CNN is proposed. Expand
Simplifying the explanation of deep neural networks with sufficient and necessary feature-sets: case of text classification
TLDR
A method to simplify the prediction explanation of One-Dimensional (1D) Convolutional Neural Networks (CNN) by identifying sufficient and necessary features-sets and an adaptation of Layer-wise Relevance Propagation for 1D-CNN is proposed. Expand
Text classification with pixel embedding
TLDR
A novel framework to understand the text by converting sentences or articles into video-like 3-dimensional tensors, which demonstrates surprisingly superior performances using the proposed model in comparison with existing methods. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 24 REFERENCES
Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
TLDR
A straightforward adaptation of CNN from image to text, a simple but new variation which employs bag-of-word conversion in the convolution layer is proposed and an extension to combine multiple convolution layers is explored for higher accuracy. Expand
A Convolutional Neural Network for Modelling Sentences
TLDR
A convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) is described that is adopted for the semantic modelling of sentences and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. Expand
Convolutional Neural Networks for Sentence Classification
TLDR
The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification, and are proposed to allow for the use of both task-specific and static vectors. Expand
Neural Network Interpretation via Fine Grained Textual Summarization
TLDR
This paper introduces the novel task of interpreting classification models using fine grained textual summarization and faithfully reflects the features learned by the model using rigorous applications like attribute based image retrieval and unsupervised text grounding. Expand
Visualizing and Understanding Convolutional Networks
TLDR
A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. Expand
Character-level Convolutional Networks for Text Classification
TLDR
This article constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results in text classification. Expand
Deep Unordered Composition Rivals Syntactic Methods for Text Classification
TLDR
This work presents a simple deep neural network that competes with and, in some cases, outperforms such models on sentiment analysis and factoid question answering tasks while taking only a fraction of the training time. Expand
Semantic Clustering and Convolutional Neural Network for Short Text Categorization
TLDR
A novel method to model short texts based on semantic clustering and convolutional neural network is proposed, which first discovers semantic cliques in embedding spaces by a fast clustering algorithm, then multi-scale semantic units are detected under the supervision of semantic clique, which introduce useful external knowledge for short texts. Expand
GloVe: Global Vectors for Word Representation
TLDR
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure. Expand
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
TLDR
A systematic evaluation of generic convolutional and recurrent architectures for sequence modeling concludes that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutionals should be regarded as a natural starting point for sequence modeled tasks. Expand
...
1
2
3
...