# Understanding Convolutional Neural Networks for Text Classification

@inproceedings{Jacovi2018UnderstandingCN, title={Understanding Convolutional Neural Networks for Text Classification}, author={Alon Jacovi and Oren Sar Shalom and Y. Goldberg}, booktitle={BlackboxNLP@EMNLP}, year={2018} }

We present an analysis into the inner workings of Convolutional Neural Networks (CNNs) for processing text. CNNs used for computer vision can be interpreted by projecting filters into image space, but for discrete sequence inputs CNNs remain a mystery. We aim to understand the method by which the networks process and classify text. We examine common hypotheses to this problem: that filters, accompanied by global max-pooling, serve as ngram detectors. We show that filters may capture several… Expand

#### Supplemental Code

#### 112 Citations

On the Interpretation of Convolutional Neural Networks for Text Classification

- Computer Science
- ECAI
- 2020

A mathematical decomposition is presented to translate the output of CNN for text classification into an ngram- level score matrix and a word-level score matrix, revealing how various parts of input sentences contribute to the final prediction quantitatively. Expand

Interpretable Text Classification Using CNN and Max-pooling

- Computer Science
- ArXiv
- 2019

This work studies the interpretability of a variant of the typical text classification model which is based on convolutional operation and max-pooling layer and evaluates the performance of the model on several classification tasks and justifies the interpretable performance with some case studies. Expand

Classifying Short Text for the Harmonized System with Convolutional Neural Networks

- 2019

Classifying goods in the hierarchical Harmonized System is a challenging problem for both humans and machines. To combat this, we propose a Convolutional Neural Network (CNN) architecture to label… Expand

Token-wise sentiment decomposition for ConvNet: Visualizing a sentiment classifier

- Computer Science
- Vis. Informatics
- 2020

This work presents a visualization technique that can be used to understand the inner workings of text-based CNN models and shows how this method can be use to generate adversarial examples and learn the shortcomings of the training data. Expand

Evaluating Recurrent Neural Network Explanations

- Computer Science, Mathematics
- BlackboxNLP@ACL
- 2019

Using the method that performed best in the authors' experiments, it is shown how specific linguistic phenomena such as the negation in sentiment analysis reflect in terms of relevance patterns, and how the relevance visualization can help to understand the misclassification of individual samples. Expand

Combining word embeddings and convolutional neural networks to detect duplicated questions

- Computer Science
- ArXiv
- 2020

This work proposes a simple approach to identifying semantically similar questions by combining the strengths of word embeddings and Convolutional Neural Networks (CNNs), and demonstrates how the cosine similarity metric can be used to effectively compare feature vectors. Expand

ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models

- Computer Science
- ArXiv
- 2021

It is shown that simple models based on CNN or LSTM as well as complex models like BERT are questionable in terms of their syntactic and semantic understanding. Expand

Simplifying the explanation of deep neural networks with sufficient and necessary feature-sets: case of text classification

- Computer Science
- ArXiv
- 2020

A method to simplify the prediction explanation of One-Dimensional (1D) Convolutional Neural Networks (CNN) by identifying sufficient and necessary features-sets and an adaptation of Layer-wise Relevance Propagation for 1D-CNN is proposed. Expand

Simplifying the explanation of deep neural networks with sufficient and necessary feature-sets: case of text classification

- Computer Science
- 2020

A method to simplify the prediction explanation of One-Dimensional (1D) Convolutional Neural Networks (CNN) by identifying sufficient and necessary features-sets and an adaptation of Layer-wise Relevance Propagation for 1D-CNN is proposed. Expand

Text classification with pixel embedding

- Computer Science
- ArXiv
- 2019

A novel framework to understand the text by converting sentences or articles into video-like 3-dimensional tensors, which demonstrates surprisingly superior performances using the proposed model in comparison with existing methods. Expand

#### References

SHOWING 1-10 OF 24 REFERENCES

Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

- Computer Science, Mathematics
- NAACL
- 2015

A straightforward adaptation of CNN from image to text, a simple but new variation which employs bag-of-word conversion in the convolution layer is proposed and an extension to combine multiple convolution layers is explored for higher accuracy. Expand

A Convolutional Neural Network for Modelling Sentences

- Computer Science
- ACL
- 2014

A convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) is described that is adopted for the semantic modelling of sentences and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. Expand

Convolutional Neural Networks for Sentence Classification

- Computer Science
- EMNLP
- 2014

The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification, and are proposed to allow for the use of both task-specific and static vectors. Expand

Neural Network Interpretation via Fine Grained Textual Summarization

- Computer Science
- ArXiv
- 2018

This paper introduces the novel task of interpreting classification models using fine grained textual summarization and faithfully reflects the features learned by the model using rigorous applications like attribute based image retrieval and unsupervised text grounding. Expand

Visualizing and Understanding Convolutional Networks

- Computer Science
- ECCV
- 2014

A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. Expand

Character-level Convolutional Networks for Text Classification

- Computer Science, Mathematics
- NIPS
- 2015

This article constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results in text classification. Expand

Deep Unordered Composition Rivals Syntactic Methods for Text Classification

- Computer Science
- ACL
- 2015

This work presents a simple deep neural network that competes with and, in some cases, outperforms such models on sentiment analysis and factoid question answering tasks while taking only a fraction of the training time. Expand

Semantic Clustering and Convolutional Neural Network for Short Text Categorization

- Computer Science
- ACL
- 2015

A novel method to model short texts based on semantic clustering and convolutional neural network is proposed, which first discovers semantic cliques in embedding spaces by a fast clustering algorithm, then multi-scale semantic units are detected under the supervision of semantic clique, which introduce useful external knowledge for short texts. Expand

GloVe: Global Vectors for Word Representation

- Computer Science
- EMNLP
- 2014

A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure. Expand

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

- Computer Science, Mathematics
- ArXiv
- 2018

A systematic evaluation of generic convolutional and recurrent architectures for sequence modeling concludes that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutionals should be regarded as a natural starting point for sequence modeled tasks. Expand