Universally Slimmable Networks and Improved Training Techniques

@article{Yu2019UniversallySN,
  title={Universally Slimmable Networks and Improved Training Techniques},
  author={J. Yu and T. Huang},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={1803-1811}
}
  • J. Yu, T. Huang
  • Published 2019
  • Computer Science
  • 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Slimmable networks are a family of neural networks that can instantly adjust the runtime width. [...] Key Method We further propose two improved training techniques for US-Nets, named the sandwich rule and inplace distillation, to enhance training process and boost testing accuracy. We show improved performance of universally slimmable MobileNet v1 and MobileNet v2 on ImageNet classification task, compared with individually trained ones and 4-switch slimmable network baselines. We also evaluate the proposed US…Expand
85 Citations
Network Slimming by Slimmable Networks: Towards One-Shot Architecture Search for Channel Numbers
  • 34
  • PDF
Dynamic Slimmable Network
  • Changlin Li, Guangrun Wang, Bing Wang, Xiaodan Liang, Zhihui Li, Xiaojun Chang
  • Computer Science
  • ArXiv
  • 2021
  • 1
  • Highly Influenced
  • PDF
MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution
  • 6
  • Highly Influenced
  • PDF
AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
  • 48
  • Highly Influenced
  • PDF
Any-Precision Deep Neural Networks
  • 5
  • PDF
GradAug: A New Regularization Method for Deep Neural Networks
  • 4
  • PDF
Learning to Learn Parameterized Classification Networks for Scalable Input Images
  • PDF
PareCO: Pareto-aware Channel Optimization for Slimmable Neural Networks
  • 2
  • Highly Influenced
  • PDF
Switchable Precision Neural Networks
  • 4
  • Highly Influenced
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 32 REFERENCES
Slimmable Neural Networks
  • 131
  • PDF
Network Slimming by Slimmable Networks: Towards One-Shot Architecture Search for Channel Numbers
  • 34
  • PDF
FitNets: Hints for Thin Deep Nets
  • 1,362
  • PDF
Data-Driven Sparse Structure Selection for Deep Neural Networks
  • 200
  • PDF
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
  • 999
  • Highly Influential
  • PDF
Deep Residual Learning for Image Recognition
  • 62,797
  • PDF
Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Execution
  • 84
  • PDF
Multi-Scale Dense Networks for Resource Efficient Image Classification
  • 229
Distilling the Knowledge in a Neural Network
  • 5,464
  • Highly Influential
  • PDF
...
1
2
3
4
...