• Corpus ID: 6947374

Interleaved Group Convolutions for Deep Neural Networks

  title={Interleaved Group Convolutions for Deep Neural Networks},
  author={Ting Zhang and Guo-Jun Qi and Bin Xiao and Jingdong Wang},
In this paper, we present a simple and modularized neural network architecture, named interleaved group convolutional neural networks (IGCNets. [] Key Result Empirical results over standard benchmarks, CIFAR-$10$, CIFAR-$100$, SVHN and ImageNet demonstrate that our networks are more efficient in using parameters and computation complexity with similar or higher accuracy.

Figures and Tables from this paper

Seesaw-Net: Convolution Neural Network With Uneven Group Convolution

Dense experiments on image classification task show that the proposed model, named Seesaw-Net, achieves state-of-the-art SOTA performance with limited computation and memory cost.

Interleaved Structured Sparse Convolutional Neural Networks

This paper presents a modularized building block, IGC-V2: interleaved structured sparse convolutions, which generalizes interleaves group convolutions to the product of more structured sparse kernels, further eliminating the redundancy.

Grouped Pointwise Convolutions Reduce Parameters in Convolutional Neural Networks

This proposal makes pointwise convolutions parameter efficient via grouping filters into parallel branches or groups, where each branch processes a fraction of the input channels, through interleaving the output of filters from different branches at intermediate layers of consecutive pointwise Convolution.

Efficient Weighted Kernel Sharing Convolutional Neural Networks

WKSC is proposed, which gathers the inputs with the same kernel, so the inputs in each group can share the same convolutional kernel, leading to a reduction of model parameters and the speedup of inference.

Weighted Channel-Wise Decomposed Convolutional Neural Networks

The Weighted Channel-wise Decomposed Convolutions (WCDC) are proposed, and the relevant networks can be called WCDC-Nets, which can achieve better performances with much fewer parameters and flop pointing computations.

CondenseNet: An Efficient DenseNet Using Learned Group Convolutions

CondenseNet is developed, a novel network architecture with unprecedented efficiency that combines dense connectivity with a novel module called learned group convolution, allowing for efficient computation in practice.

Training Better CNNs Requires to Rethink ReLU

The proportional module with N:M (N$>$M) convolution and ReLU ratio can help networks acquire the better performance, through the analysis of a simple Leaky ReLU model and the experimental results verify that the superiority of the proportional module is found.

Rethinking ReLU to Train Better CNNs

This paper proposes a proportional module to keep the ratio between convolution and ReLU amount to be N:m (n>m), which can be applied in almost all networks with no extra computational cost to improve the performance.

Rethink ReLU to Training Better CNNs

This paper proposes a proportional module to keep the ratio between convolution and ReLU amount to be N:M (N>M) and achieves better performance on different benchmarks with different network architectures.

Deep ensemble network based on multi-path fusion

Experimental results on the standard recognition tasks have shown that the proposed deep ensemble network achieves better classification performance and has superior generalization ability compared to the original residual networks.



Flattened Convolutional Neural Networks for Feedforward Acceleration

Flattened convolutional neural networks that are designed for fast feedforward execution are presented and it is found that the flattened layer can effectively substitute for the 3D filters without loss of accuracy.

Densely Connected Convolutional Networks

The Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion, and has several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters.

Striving for Simplicity: The All Convolutional Net

It is found that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks.

Network In Network

With enhanced local modeling via the micro network, the proposed deep network structure NIN is able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers.

The Power of Sparsity in Convolutional Neural Networks

2D convolution is generalized to use a channel-wise sparse connection structure and it is shown that this leads to significantly better results than the baseline approach for large networks including VGG and Inception V3.

Aggregated Residual Transformations for Deep Neural Networks

On the ImageNet-1K dataset, it is empirically show that even under the restricted condition of maintaining complexity, increasing cardinality is able to improve classification accuracy and is more effective than going deeper or wider when the authors increase the capacity.

ImageNet classification with deep convolutional neural networks

A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.

Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups

We propose a new method for creating computationally efficient and compact convolutional neural networks (CNNs) using a novel sparse connection structure that resembles a tree root. This allows a

Speeding up Convolutional Neural Networks with Low Rank Expansions

Two simple schemes for drastically speeding up convolutional neural networks are presented, achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain.

On the Connection of Deep Fusion to Ensembling

A new deeply-fused network is developed that combines two networks in a merge-and-run fusion manner that is less deep than a ResNet but yields an ensemble of the same number of more-capable component networks, thus improving the classification accuracy.