HRank: Filter Pruning Using High-Rank Feature Map

@article{Lin2020HRankFP,
  title={HRank: Filter Pruning Using High-Rank Feature Map},
  author={Mingbao Lin and Rongrong Ji and Yan Wang and Yichen Zhang and Baochang Zhang and Yonghong Tian and Ling Shao},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={1526-1535}
}
  • Mingbao LinRongrong Ji L. Shao
  • Published 24 February 2020
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Neural network pruning offers a promising prospect to facilitate deploying deep neural networks on resource-limited devices. However, existing methods are still challenged by the training inefficiency and labor cost in pruning designs, due to missing theoretical guidance of non-salient network components. In this paper, we propose a novel filter pruning method by exploring the High Rank of feature maps (HRank). Our HRank is inspired by the discovery that the average rank of multiple feature… 
[PDF] Semantic Reader
341 Citations
Highly Influential Citations
74
Background Citations
147
Methods Citations
157
Results Citations
15

Figures and Tables from this paper

341 Citations

FPAR: Filter Pruning Via Attention and Rank Enhancement

This work proposes a novel filter pruning method, called Filter Pruning via Attention and Rank Enhancement (FPAR), based on the attention mechanism and rank of feature maps, and demonstrates that it outperforms the most advanced methods with similar accuracy.

CSHE: network pruning by using cluster similarity and matrix eigenvalues

A novel filter pruning method that combines convolution filters and feature maps information for convolutional neural network compression, namely network pruning by using cluster similarity and large eigenvalues (CSHE).

Filter pruning with a feature map entropy importance criterion for convolution neural networks compressing

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

A new mathematical formulation is presented to accurately and efficiently quantify the feature-map discriminativeness, which gives rise to a novel criterion,Discriminant Information (DI), and a DI-based greedy pruning algorithm and structure distillation technique to automatically decide the pruned structure that satisfies certain resource budget, which is a common requirement in reality.

Towards efficient filter pruning via topology

With the development of deep neural networks, compressing and accelerating deep neural networks without performance deterioration has become a research hotspot. Among all kinds of network compression…

Filter Pruning via Measuring Feature Map Information

A new pruning method is proposed, which reflects the importance of filters by exploring the information in the feature maps, which is based on the premise that the more information there is, more important the feature map is.

Efficient Network Pruning via Feature Selection

This paper proposes a network pruning method from a feature selection perspective, and achieves impressive efficiency by resulting in less accuracy drop with similar pruning ratios as others, especially in terms of the parameter compression ratio.

Network Compression via Central Filter

This paper proposes a novel filter pruning method—Central Filter (CF), which suggests that a filter is approximately equal to a set of other filters after appropriate adjustments, based on the discovery that the average similarity between feature maps changes very little, regardless of the number of input images.

Network Pruning via Feature Shift Minimization

A novel Feature Shift Minimization (FSM) method to compress CNN models, which evalu-ates the feature shift by converging the information of both features andters, and presents a distribution-optimization algorithm to compensate for the accuracy loss and improve the network compression efficiency.

Linearly Replaceable Filters for Deep Network Channel Pruning

A novel channel pruning method, Linearly Replaceable Filter (LRF), is proposed, which suggests that a filter that can be approximated by the linear combination of other filters is replaceable.
...

References

SHOWING 1-10 OF 37 REFERENCES

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration

Unlike previous methods, FPGM compresses CNN models by pruning filters with redundancy, rather than those with“relatively less” importance, and when applied to two image classification benchmarks, the method validates its usefulness and strengths.

Accelerating Convolutional Networks via Global & Dynamic Filter Pruning

This paper proposes a novel global & dynamic pruning (GDP) scheme to prune redundant filters for CNN acceleration that achieves superior performance to accelerate several cutting-edge CNNs on the ILSVRC 2012 benchmark.

Pruning Convolutional Neural Networks for Resource Efficient Inference

It is shown that pruning can lead to more than 10x theoretical (5x practical) reduction in adapted 3D-convolutional filters with a small drop in accuracy in a recurrent gesture classifier.

Variational Convolutional Neural Network Pruning

Variational technique is introduced to estimate distribution of a newly proposed parameter, called channel saliency, based on which redundant channels can be removed from model via a simple criterion, and results in significant size reduction and computation saving.

Pruning Filters for Efficient ConvNets

This work presents an acceleration method for CNNs, where it is shown that even simple filter pruning techniques can reduce inference costs for VGG-16 and ResNet-110 by up to 38% on CIFAR10 while regaining close to the original accuracy by retraining the networks.

ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression

ThiNet is proposed, an efficient and unified framework to simultaneously accelerate and compress CNN models in both training and inference stages, and it is revealed that it needs to prune filters based on statistics information computed from its next layer, not the current layer, which differentiates ThiNet from existing methods.

NISP: Pruning Networks Using Neuron Importance Score Propagation

  • Ruichi YuAng Li L. Davis
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
The Neuron Importance Score Propagation (NISP) algorithm is proposed to propagate the importance scores of final responses to every neuron in the network and is evaluated on several datasets with multiple CNN models and demonstrated to achieve significant acceleration and compression with negligible accuracy loss.

Data-Driven Sparse Structure Selection for Deep Neural Networks

A simple and effective framework to learn and prune deep models in an end-to-end manner by adding sparsity regularizations on factors, and solving the optimization problem by a modified stochastic Accelerated Proximal Gradient (APG) method.

Filter Sketch for Network Pruning

FilterSketch encodes the second-order information of pretrained weights, which enables the representation capacity of pruned networks to be recovered with a simple fine-tuning procedure, leading to a several-orders-of-magnitude reduction of time cost in the optimization of pruning.

Channel Pruning via Automatic Structure Search

This paper proposes a new channel pruning method based on artificial bee colony algorithm (ABC), dubbed as ABCPruner, which aims to efficiently find optimal pruned structure, i.e., channel number in each layer, rather than selecting "important" channels as previous works did.