Densely Connected CNN with Multi-scale Feature Attention for Text Classification

@inproceedings{Wang2018DenselyCC,
  title={Densely Connected CNN with Multi-scale Feature Attention for Text Classification},
  author={Shiyao Wang and Minlie Huang and Zhidong Deng},
  booktitle={IJCAI},
  year={2018}
}
Text classification is a fundamental problem in natural language processing. [] Key Method The dense connections build short-cut paths between upstream and downstream convolutional blocks, which enable the model to compose features of larger scale from those of smaller scale, and thus produce variable n-gram features. Furthermore, a multi-scale feature attention is developed to adaptively select multi-scale features for classification. Extensive experiments demonstrate that our model obtains competitive…
View via Publisher
ijcai.org
95 Citations
Highly Influential Citations
12
Background Citations
27
Methods Citations
35

Figures and Tables from this paper

95 Citations

Large-Scale Text Classification Using Scope-Based Convolutional Neural Network: A Deep Learning Approach
TLDR
A large-scale scope-based convolutional neural network (LSS-CNN), which is based on scope convolution, aggregation optimization, and max pooling operation, which can achieve both effectiveness and good scalability on big text data.
Hierarchical Gated Convolutional Networks with Multi-Head Attention for Text Classification
  • Haizhou Du, Jingu Qian
  • Computer Science
    2018 5th International Conference on Systems and Informatics (ICSAI)
  • 2018
TLDR
H hierarchical gated convolutional networks with multi-head attention with hierarchical structure the same as the hierarchical structure of documents that has word-level and sentence-level is proposed which reduces computational cost through its two distinctive characteristics to save considerable model parameters.
Text Classification Using Scope Based Convolutional Neural Network
  • Jiaying Wang, Jing Shan, Yaxin Li, Yi Ren, Duo Zhai, Jinling Bao
  • Computer Science
    2019 IEEE International Conferences on Ubiquitous Computing & Communications (IUCC) and Data Science and Computational Intelligence (DSCI) and Smart Computing, Networking and Services (SmartCNS)
  • 2019
TLDR
A scope based convolutional neural network (SCNN) that can capture deeper local information which is hidden from windows based CNN and utilize max pooling to keep the most valuable information.
Adaptive Feature Extractor of Global Representation and Local Semantics for Text Classification
TLDR
An adaptive approach to automatically determine the contribution degree of each model to classification, according to specific structure and grammar information of the text, to better extract variable-size features of a word.
A Discriminative Convolutional Neural Network with Context-aware Attention
TLDR
This article proposes a discriminative CNN with context-aware attention to solve the challenges of vanilla CNN, and mainly encourages discrimination across different filters via maximizing their earth mover distances and estimates the salience of feature candidates by considering the relation between context features.
Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism
TLDR
An attention-based Bi-LSTM+CNN hybrid model that capitalize on the advantages of LSTM and CNN with an additional attention mechanism is proposed that produces more accurate classification results, as well as higher recall and F1 scores, than individual multi-layer perceptron (MLP), CNN or L STM models as the hybrid models.
Hierarchical Text-Label Integrated Attention Network for Document Classification
TLDR
HLAN achieves competitive results over the compared strong baseline methods on 4 out of 5 benchmark datasets, which verifies the effectiveness of HLAN for document classification, in terms of both accuracy and memory requirement.
Weighted N-grams CNN for Text Classification
TLDR
The experimental results show that the proposed weighted n-grams CNN model exhibits comparable or even superior performance with other popular and latest CNN models on five datasets in text classification.
Dynamically Route Hierarchical Structure Representation to Attentive Capsule for Text Classification
TLDR
This paper proposes a novel architecture that dynamically routes hierarchical structure feature to attentive capsule, named HAC, and adopts intermediate information of a well-designed deep dilated CNN to form hierarchical structure features.
Multi-label text classification via hierarchical Transformer-CNN
TLDR
A hierarchical Transformer-CNN model is constructed to capture the semantic information of different levels of the text at the word and sentence levels using multi-headed self-attention mechanism, and a sentence convolutional neural network was used to extract key semantic features.
...
...

References

SHOWING 1-10 OF 41 REFERENCES
Deep Pyramid Convolutional Neural Networks for Text Categorization
TLDR
A low-complexity word-level deep convolutional neural network architecture for text categorization that can efficiently represent long-range associations in text and outperforms the previous best models on six benchmark datasets for sentiment classification and topic categorization.
Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
TLDR
A straightforward adaptation of CNN from image to text, a simple but new variation which employs bag-of-word conversion in the convolution layer is proposed and an extension to combine multiple convolution layers is explored for higher accuracy.
Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification
TLDR
This paper proposes a framework based on convolutional neural networks that combines explicit and implicit representations of short text for classification and shows that the proposed method significantly outperforms state-of-the-art methods.
Molding CNNs for text: non-linear, non-consecutive convolutions
TLDR
This work revise the temporal convolution operation in CNNs to better adapt it to text processing by appealing to tensor algebra and using low-rank n-gram tensors to directly exploit interactions between words already at the convolution stage.
Character-level Convolutional Networks for Text Classification
TLDR
This article constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results in text classification.
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
TLDR
This work presents a general Attention Based Convolutional Neural Network (ABCNN) for modeling a pair of sentences and proposes three attention schemes that integrate mutual influence between sentences into CNNs; thus, the representation of each sentence takes into consideration its counterpart.
A Convolutional Neural Network for Modelling Sentences
TLDR
A convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) is described that is adopted for the semantic modelling of sentences and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations.
Very Deep Convolutional Networks for Natural Language Processing
TLDR
This work presents a new architecture for text processing which operates directly on the character level and uses only small convolutions and pooling operations, and is able to show that the performance of this model increases with the depth.
Encoding Syntactic Knowledge in Neural Networks for Sentiment Classification
TLDR
This article proposes to learn tag-specific composition functions and tag embeddings in recursive neural networks, and proposes to utilize POS tags to control the gates of tree-structured LSTM networks.
Densely Connected Convolutional Networks
TLDR
The Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion, and has several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters.
...
...