Deep Networks with Stochastic Depth

@inproceedings{Huang2016DeepNW,
  title={Deep Networks with Stochastic Depth},
  author={Gao Huang and Yu Sun and Zhuang Liu and Daniel Sedra and Kilian Q. Weinberger},
  booktitle={ECCV},
  year={2016}
}
  • Gao Huang, Yu Sun, +2 authors Kilian Q. Weinberger
  • Published in ECCV 2016
  • Computer Science
  • Very deep convolutional networks with hundreds of layers have led to significant reductions in error on competitive benchmarks. Although the unmatched expressiveness of the many layers can be highly desirable at test time, training very deep networks comes with its own set of challenges. The gradients can vanish, the forward flow often diminishes, and the training time can be painfully slow. To address these problems, we propose stochastic depth, a training procedure that enables the seemingly… CONTINUE READING
    914 Citations
    Wide Residual Networks
    • 2,667
    • Highly Influenced
    • PDF
    On the importance of network architecture in training very deep neural networks
    • 3
    Going Deeper With Neural Networks Without Skip Connections
    • 1
    Depth Dropout: Efficient Training of Residual Convolutional Neural Networks
    • J. Guo, Stephen Gould
    • Computer Science
    • 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA)
    • 2016
    • 9
    • PDF
    Weighted residuals for very deep networks
    • Falong Shen, Gang Zeng
    • Computer Science
    • 2016 3rd International Conference on Systems and Informatics (ICSAI)
    • 2016
    • 18
    • PDF
    Deep networks with stochastic depth for acoustic modelling
    • 8
    The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs
    • 9
    • PDF
    Improving the Capacity of Very Deep Networks with Maxout Units
    • 7
    • PDF

    References

    SHOWING 1-10 OF 39 REFERENCES
    Deep Residual Learning for Image Recognition
    • 57,985
    • Highly Influential
    • PDF
    Deeply-Supervised Nets
    • 1,137
    • PDF
    Gradual DropIn of Layers to Train Very Deep Neural Networks
    • 22
    • PDF
    On the importance of initialization and momentum in deep learning
    • 2,682
    • PDF
    Very Deep Convolutional Networks for Large-Scale Image Recognition
    • 43,733
    • PDF
    Striving for Simplicity: The All Convolutional Net
    • 2,376
    • PDF
    Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
    • 20,832
    • Highly Influential
    • PDF
    Understanding the difficulty of training deep feedforward neural networks
    • 9,246
    • PDF
    Dropout: a simple way to prevent neural networks from overfitting
    • 20,930
    • PDF