Corpus ID: 195346430

Deep Net Triage: Assessing the Criticality of Network Layers by Structural Compression

  title={Deep Net Triage: Assessing the Criticality of Network Layers by Structural Compression},
  author={Theodore S. Nowak and Jason J. Corso},
Deep network compression seeks to reduce the number of parameters in the network while maintaining a certain level of performance. Deep network distillation seeks to train a smaller network that matches soft-max performance of a larger network. While both regimes have led to impressive performance for their respective goals, neither provide insight into the importance of a given layer in the original model, which is useful if we are to improve our understanding of these highly parameterized… Expand
3 Citations
Knowledge Distillation: A Survey
  • 22
  • PDF
ResKD: Residual-Guided Knowledge Distillation
  • 1
  • PDF
Progressive Blockwise Knowledge Distillation for Neural Network Acceleration
  • 21
  • Highly Influenced
  • PDF


FitNets: Hints for Thin Deep Nets
  • 1,353
  • PDF
Greedy Layer-Wise Training of Deep Networks
  • 1,502
Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding
  • 4,320
  • PDF
Distilling the Knowledge in a Neural Network
  • 5,436
  • PDF
Deep Residual Learning for Image Recognition
  • 62,552
  • PDF
Very Deep Convolutional Networks for Large-Scale Image Recognition
  • 46,192
  • PDF
Understanding the difficulty of training deep feedforward neural networks
  • 9,658
  • PDF
Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
  • 139
  • PDF