• Corpus ID: 239050528

Class-Discriminative CNN Compression

@article{Liu2021ClassDiscriminativeCC,
  title={Class-Discriminative CNN Compression},
  author={Yuchen Liu and David Wentzlaff and S. Y. Kung},
  journal={ArXiv},
  year={2021},
  volume={abs/2110.10864}
}
Compressing convolutional neural networks (CNNs) by pruning and distillation has received ever-increasing focus in the community. In particular, designing a classdiscrimination based approach would be desired as it fits seamlessly with the CNNs training objective. In this paper, we propose class-discriminative compression (CDC), which injects class discrimination in both pruning and distillation to facilitate the CNNs training goal. We first study the effectiveness of a group of discriminant… 

References

SHOWING 1-10 OF 62 REFERENCES
Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression
  • Yuchao Li, Shaohui Lin, +5 authors R. Ji
  • Computer Science
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
KSE is capable of simultaneously compressing each layer in an efficient way, which is significantly faster compared to previous data-driven feature map pruning methods, and significantly outperforms state-of-the-art methods.
Towards Compact CNNs via Collaborative Compression
TLDR
A Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models by simultaneously learning the model sparsity and low-rankness and proposes multi-step heuristic compression to remove redundant compression units step-by-step.
Learning Efficient Object Detection Models with Knowledge Distillation
TLDR
This work proposes a new framework to learn compact and fast object detection networks with improved accuracy using knowledge distillation and hint learning and shows consistent improvement in accuracy-speed trade-offs for modern multi-class detection models.
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
TLDR
ThiNet is proposed, an efficient and unified framework to simultaneously accelerate and compress CNN models in both training and inference stages, and it is revealed that it needs to prune filters based on statistics information computed from its next layer, not the current layer, which differentiates ThiNet from existing methods.
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
  • Shaohui Lin, R. Ji, +5 authors D. Doermann
  • Computer Science
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
This paper proposes an effective structured pruning approach that jointly prunes filters as well as other structures in an end-to-end manner and effectively solves the optimization problem by generative adversarial learning (GAL), which learns a sparse soft mask in a label-free and an end to end manner.
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks
TLDR
This work proposes a global filter pruning algorithm called Gate Decorator, which transforms a vanilla CNN module by multiplying its output by the channel-wise scaling factors (i.e. gate), and proposes an iterative pruning framework called Tick-Tock to improve pruning accuracy.
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
TLDR
Unlike previous methods, FPGM compresses CNN models by pruning filters with redundancy, rather than those with“relatively less” importance, and when applied to two image classification benchmarks, the method validates its usefulness and strengths.
Learning Efficient Convolutional Networks through Network Slimming
TLDR
The approach is called network slimming, which takes wide and large networks as input models, but during training insignificant channels are automatically identified and pruned afterwards, yielding thin and compact models with comparable accuracy.
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers
TLDR
This paper proposes a channel pruning technique for accelerating the computations of deep convolutional neural networks (CNNs) that focuses on direct simplification of the channel-to-channel computation graph of a CNN without the need of performing a computationally difficult and not-always-useful task.
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning
TLDR
This paper proposes a novel global & dynamic pruning (GDP) scheme to prune redundant filters for CNN acceleration that achieves superior performance to accelerate several cutting-edge CNNs on the ILSVRC 2012 benchmark.
...
1
2
3
4
5
...