Corpus ID: 195346430

Deep Net Triage: Assessing the Criticality of Network Layers by Structural Compression

@article{Nowak2018DeepNT,
  title={Deep Net Triage: Assessing the Criticality of Network Layers by Structural Compression},
  author={Theodore S. Nowak and Jason J. Corso},
  journal={ArXiv},
  year={2018},
  volume={abs/1801.04651}
}
Deep network compression seeks to reduce the number of parameters in the network while maintaining a certain level of performance. Deep network distillation seeks to train a smaller network that matches soft-max performance of a larger network. While both regimes have led to impressive performance for their respective goals, neither provide insight into the importance of a given layer in the original model, which is useful if we are to improve our understanding of these highly parameterized… Expand
3 Citations
Knowledge Distillation: A Survey
  • 22
  • PDF
ResKD: Residual-Guided Knowledge Distillation
  • 1
  • PDF
Progressive Blockwise Knowledge Distillation for Neural Network Acceleration
  • 21
  • Highly Influenced
  • PDF

References

SHOWING 1-10 OF 17 REFERENCES
FitNets: Hints for Thin Deep Nets
  • 1,353
  • PDF
Greedy Layer-Wise Training of Deep Networks
  • 1,502
Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding
  • 4,320
  • PDF
Distilling the Knowledge in a Neural Network
  • 5,436
  • PDF
Deep Residual Learning for Image Recognition
  • 62,552
  • PDF
Very Deep Convolutional Networks for Large-Scale Image Recognition
  • 46,192
  • PDF
Understanding the difficulty of training deep feedforward neural networks
  • 9,658
  • PDF
Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
  • 139
  • PDF
...
1
2
...