Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

@article{He2015DelvingDI,
  title={Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification},
  author={Kaiming He and X. Zhang and Shaoqing Ren and Jian Sun},
  journal={2015 IEEE International Conference on Computer Vision (ICCV)},
  year={2015},
  pages={1026-1034}
}
  • Kaiming He, X. Zhang, +1 author Jian Sun
  • Published 2015
  • Computer Science
  • 2015 IEEE International Conference on Computer Vision (ICCV)
  • Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. [...] Key Method PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on the learnable activation and advanced…Expand Abstract
    8,399 Citations
    Empirical Evaluation of Rectified Activations in Convolutional Network
    • 1,114
    • PDF
    Overcoming Overfitting and Large Weight Update Problem in Linear Rectifiers: Thresholded Exponential Rectified Linear Units
    • V. Pandey
    • Computer Science, Mathematics
    • ArXiv
    • 2020
    • 1
    • Highly Influenced
    • PDF
    A Deep Convolutional Neural Network with Selection Units for Super-Resolution
    • Jae-Seok Choi, M. Kim
    • Computer Science
    • 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
    • 2017
    • 38
    • PDF
    FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks
    • Suo Qiu, B. Cai
    • Computer Science
    • 2018 24th International Conference on Pattern Recognition (ICPR)
    • 2018
    • 17
    • Highly Influenced
    • PDF
    Deep Transfer Learning for Art Classification Problems
    • 11
    • PDF
    Beyond ImageNet: Deep Learning in Industrial Practice
    • 6
    • PDF
    Deep neural networks with Elastic Rectified Linear Units for object recognition
    • 29
    • Highly Influenced
    Shallow and wide fractional max-pooling network for image classification
    • 3
    Very deep convolutional neural network based image classification using small training sample size
    • Shuying Liu, W. Deng
    • Computer Science
    • 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR)
    • 2015
    • 184
    Global-connected network with generalized ReLU activation
    • 5

    References

    SHOWING 1-10 OF 46 REFERENCES
    Deeply-Supervised Nets
    • 1,103
    • PDF
    ImageNet classification with deep convolutional neural networks
    • 55,120
    • Highly Influential
    • PDF
    Convolutional neural networks at constrained time cost
    • Kaiming He, Jian Sun
    • Computer Science
    • 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
    • 2015
    • 583
    • PDF
    Return of the Devil in the Details: Delving Deep into Convolutional Nets
    • 2,662
    • PDF
    Going deeper with convolutions
    • 20,857
    • Highly Influential
    • PDF
    Very Deep Convolutional Networks for Large-Scale Image Recognition
    • 41,291
    • Highly Influential
    • PDF
    Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
    • 19,722
    • PDF
    DeepFace: Closing the Gap to Human-Level Performance in Face Verification
    • 4,103
    • PDF
    Some Improvements on Deep Convolutional Neural Network Based Image Classification
    • 279
    • PDF
    On rectified linear units for speech processing
    • 419
    • PDF