Deep Networks with Stochastic Depth
@inproceedings{Huang2016DeepNW, title={Deep Networks with Stochastic Depth}, author={Gao Huang and Yu Sun and Zhuang Liu and Daniel Sedra and Kilian Q. Weinberger}, booktitle={ECCV}, year={2016} }
Very deep convolutional networks with hundreds of layers have led to significant reductions in error on competitive benchmarks. Although the unmatched expressiveness of the many layers can be highly desirable at test time, training very deep networks comes with its own set of challenges. The gradients can vanish, the forward flow often diminishes, and the training time can be painfully slow. To address these problems, we propose stochastic depth, a training procedure that enables the seemingly… CONTINUE READING
Supplemental Content
Github Repo
Via Papers with Code
Sandbox for training convolutional networks for computer vision
Figures, Tables, and Topics from this paper
Paper Mentions
914 Citations
On the importance of network architecture in training very deep neural networks
- Computer Science
- 2016 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC)
- 2016
- 3
Going Deeper With Neural Networks Without Skip Connections
- Computer Science
- 2020 IEEE International Conference on Image Processing (ICIP)
- 2020
- 1
Training Very Deep Networks via Residual Learning with Stochastic Input Shortcut Connections
- Computer Science
- ICONIP
- 2017
- 11
- PDF
Depth Dropout: Efficient Training of Residual Convolutional Neural Networks
- Computer Science
- 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA)
- 2016
- 9
- PDF
Weighted residuals for very deep networks
- Computer Science
- 2016 3rd International Conference on Systems and Informatics (ICSAI)
- 2016
- 18
- PDF
Deep networks with stochastic depth for acoustic modelling
- Computer Science
- 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA)
- 2016
- 8
The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs
- Computer Science, Geology
- ArXiv
- 2016
- 9
- PDF
Improving the Capacity of Very Deep Networks with Maxout Units
- Computer Science
- 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2018
- 7
- PDF
References
SHOWING 1-10 OF 39 REFERENCES
Deep Residual Learning for Image Recognition
- Computer Science
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2016
- 57,985
- Highly Influential
- PDF
Gradual DropIn of Layers to Train Very Deep Neural Networks
- Computer Science
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2016
- 22
- PDF
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Computer Science
- ICML
- 2015
- 20,832
- Highly Influential
- PDF
Understanding the difficulty of training deep feedforward neural networks
- Computer Science, Mathematics
- AISTATS
- 2010
- 9,246
- PDF
ImageNet classification with deep convolutional neural networks
- Computer Science
- Commun. ACM
- 2012
- 58,386
- PDF
Dropout: a simple way to prevent neural networks from overfitting
- Computer Science
- J. Mach. Learn. Res.
- 2014
- 20,930
- PDF