Deep Residual Learning for Image Recognition

@article{He2016DeepRL,
  title={Deep Residual Learning for Image Recognition},
  author={Kaiming He and X. Zhang and Shaoqing Ren and Jian Sun},
  journal={2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2016},
  pages={770-778}
}
  • Kaiming He, X. Zhang, +1 author Jian Sun
  • Published 10 December 2015
  • Computer Science
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On… Expand
Study of Residual Networks for Image Recognition
TLDR
The findings illustrate that ResNets are more prone to overfitting despite their higher accuracy, and several methods to prevent overfitting such as adding dropout layers and stochastic augmentation of the training dataset has been studied in this work. Expand
Shallow Learning For Deep Networks
Shallow supervised 1-hidden layer neural networks have a number of favorable properties that make them easier to interpret, analyze, and optimize than their deep counterparts, but lack theirExpand
Hyperparameter Optimization for Deep Residual Learning in Image Classification
  • Abbas Jafar, Myungho Lee
  • Computer Science
  • 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)
  • 2020
TLDR
A manual search approach for ResNet architecture is built by altering the traditional data augmentation proposed by the previous approaches and significantly reduces the computing time compared with the previous automatic approaches. Expand
Deep Residual Networks with Exponential Linear Unit
TLDR
This paper proposes to replace the combination of ReLU and Batch Normalization with Exponential Linear Unit (ELU) in Residual Networks, and shows that this not only speeds up the learning behavior in Residine Networks, but also improves the classification performance as the depth increases. Expand
On the importance of network architecture in training very deep neural networks
TLDR
This paper investigates the residual module in great extent by analyzing the structure ordering of different blocks and modify them one by one to achieve lower test error on CIFAR-10 dataset, and proposes a random-jump scheme to skip some residual connections during training. Expand
Convolutional Recurrent Neural Networks for Better Image Understanding
TLDR
This work introduces a recurrent unit able to keep and process spatial information throughout the network, and shows that its approach based on higher resolution input is better able to detect details of the images such as the precise number of objects, and the presence of smaller objects, while being less sensitive to biases in the label distribution of the training set. Expand
Weighted residuals for very deep networks
  • Falong Shen, Gang Zeng
  • Computer Science
  • 2016 3rd International Conference on Systems and Informatics (ICSAI)
  • 2016
TLDR
A weighted residual network is introduced to address the incompatibility between ReLU and element-wise addition and the deep network initialization problem and is able to learn to combine residuals from different layers effectively and efficiently. Expand
Improved Residual Networks for Image and Video Recognition
TLDR
The proposed improvements address all three main components of a ResNet: the flow of information through the network layers, the residual building block, and the projection shortcut, and are able to show consistent improvements in accuracy and learning convergence over the baseline. Expand
Greedy Layerwise Learning Can Scale to ImageNet
TLDR
This work uses 1-hidden layer learning problems to sequentially build deep networks layer by layer, which can inherit properties from shallow networks, and obtains an 11-layer network that exceeds several members of the VGG model family on ImageNet, and can train a VGG-11 model to the same accuracy as end-to-end learning. Expand
Energy-Efficient and Improved Image Recognition with Conditional Deep Learning
TLDR
Conditional Deep Learning is proposed, where the convolutional layer features are used to identify the variability in the difficulty of input instances and conditionally activate the deeper layers of the network, resulting in improved classification over state-of-the-art baseline networks. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 62 REFERENCES
Deep Residual Learning for Image Recognition Supplementary Materials
In this section we introduce our detection method based on the baseline Faster R-CNN [6] system. The models are initialized by the ImageNet classification models, and then fine-tuned on the objectExpand
Very Deep Convolutional Networks for Large-Scale Image Recognition
TLDR
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. Expand
ImageNet classification with deep convolutional neural networks
TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective. Expand
Deeply-Supervised Nets
TLDR
The proposed deeply-supervised nets (DSN) method simultaneously minimizes classification error while making the learning process of hidden layers direct and transparent, and extends techniques from stochastic gradient methods to analyze the algorithm. Expand
Understanding the difficulty of training deep feedforward neural networks
TLDR
The objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future. Expand
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
TLDR
This work proposes a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit and derives a robust initialization method that particularly considers the rectifier nonlinearities. Expand
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
TLDR
Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Expand
Network In Network
TLDR
With enhanced local modeling via the micro network, the proposed deep network structure NIN is able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers. Expand
Learning Multiple Layers of Features from Tiny Images
TLDR
It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network. Expand
Deep Learning Made Easier by Linear Transformations in Perceptrons
TLDR
The usefulness of the transformations are confirmed, which make basic stochastic gradient learning competitive with state-of-the-art learning algorithms in speed and that they seem also to help find solutions that generalize better. Expand
...
1
2
3
4
5
...