Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

@article{He2015DelvingDI,
  title={Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification},
  author={Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun},
  journal={2015 IEEE International Conference on Computer Vision (ICCV)},
  year={2015},
  pages={1026-1034}
}
Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities… CONTINUE READING

Figures, Tables, Results, and Topics from this paper.

Key Quantitative Results

  • Based on the learnable activation and advanced initialization, we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset.

Citations

Publications citing this paper.
SHOWING 1-10 OF 3,972 CITATIONS

Enhancing Quality for HEVC Compressed Videos

  • IEEE Transactions on Circuits and Systems for Video Technology
  • 2019
VIEW 21 EXCERPTS
CITES METHODS
HIGHLY INFLUENCED

S2-LBI: Stochastic Split Linearized Bregman Iterations for Parsimonious Deep Learning

  • ArXiv
  • 2019
VIEW 10 EXCERPTS
CITES RESULTS, BACKGROUND & METHODS
HIGHLY INFLUENCED

Sample Variance Decay in Randomly Initialized ReLU Networks

VIEW 5 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks

Sanghyun Hong, Pietro Frigo, Yigitcan Kaya, Cristiano Giuffrida, Tudor Dumitras
  • USENIX Security Symposium
  • 2019
VIEW 8 EXCERPTS
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2014
2019

CITATION STATISTICS

  • 401 Highly Influenced Citations

  • Averaged 1,104 Citations per year from 2017 through 2019

References

Publications referenced by this paper.
SHOWING 1-10 OF 35 REFERENCES

Convolutional neural networks at constrained time cost

  • 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2014
VIEW 4 EXCERPTS

ImageNet Large Scale Visual Recognition Challenge

  • International Journal of Computer Vision
  • 2014
VIEW 10 EXCERPTS
HIGHLY INFLUENTIAL

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2014
VIEW 9 EXCERPTS

Going deeper with convolutions

  • 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2014
VIEW 5 EXCERPTS
HIGHLY INFLUENTIAL

On rectified linear units for speech processing

  • 2013 IEEE International Conference on Acoustics, Speech and Signal Processing
  • 2013
VIEW 6 EXCERPTS
HIGHLY INFLUENTIAL