All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation

@article{Xie2017AllYN,
  title={All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation},
  author={Di Xie and Jiang Xiong and Shiliang Pu},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2017},
  pages={5075-5084}
}
Deep neural network is difficult to train and this predicament becomes worse as the depth increases. The essence of this problem exists in the magnitude of backpropagated errors that will result in gradient vanishing or exploding phenomenon. We show that a variant of regularizer which utilizes orthonormality among different filter banks can alleviate this problem. Moreover, we design a backward error modulation mechanism based on the quasi-isometry assumption between two consecutive parametric… CONTINUE READING
Highly Cited
This paper has 28 citations. REVIEW CITATIONS
Related Discussions
This paper has been referenced on Twitter 21 times. VIEW TWEETS

Citations

Publications citing this paper.

References

Publications referenced by this paper.
Showing 1-10 of 45 references

ImageNet Large Scale Visual Recognition Challenge

International Journal of Computer Vision • 2015
View 6 Excerpts
Highly Influenced

Deep Residual Learning for Image Recognition

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2016
View 5 Excerpts
Highly Influenced

Similar Papers

Loading similar papers…