Corpus ID: 1289873

Deeply-Supervised Nets

@article{Lee2015DeeplySupervisedN,
  title={Deeply-Supervised Nets},
  author={Chen-Yu Lee and Saining Xie and Patrick W. Gallagher and Z. Zhang and Zhuowen Tu},
  journal={ArXiv},
  year={2015},
  volume={abs/1409.5185}
}
Our proposed deeply-supervised nets (DSN) method simultaneously minimizes classification error while making the learning process of hidden layers direct and transparent. [...] Key Method We introduce "companion objective" to the individual hidden layers, in addition to the overall objective at the output layer (a different strategy to layer-wise pre-training). We extend techniques from stochastic gradient methods to analyze our algorithm. The advantage of our method is evident and our experimental result on…Expand
Multi-way backpropagation for training compact deep neural networks
TLDR
This paper first investigates the supervision vanishing issue in existing backpropagation (BP) methods and proposes to address it via an effective method, called Multi-way BP (MW-BP), which relies on multiple auxiliary losses added to the intermediate layers of the network. Expand
Improving deep neural network performance by integrating kernelized Min-Max objective
TLDR
Extensive experimental results demonstrate that the integration of kernelized Min-Max objective in the training of DNN models can achieve better results compared to state-of-the-art models, without incurring additional model complexity. Expand
On the importance of network architecture in training very deep neural networks
TLDR
This paper investigates the residual module in great extent by analyzing the structure ordering of different blocks and modify them one by one to achieve lower test error on CIFAR-10 dataset, and proposes a random-jump scheme to skip some residual connections during training. Expand
Novel CNN architecture with residual learning and deep supervision for large-scale scene image categorization
TLDR
Results of the experiments support the hypothesis that adding the residual connections to the CNDS will enhance the accuracy of the network and improve upon other recently introduced state of the art networks both in terms of top-1 and top-5 classification accuracy. Expand
Deep Pyramidal Residual Networks
TLDR
This research gradually increases the feature map dimension at all units to involve as many locations as possible in the network architecture and proposes a novel residual unit capable of further improving the classification accuracy with the new network architecture. Expand
Greedy Layerwise Learning Can Scale to ImageNet
TLDR
This work uses 1-hidden layer learning problems to sequentially build deep networks layer by layer, which can inherit properties from shallow networks, and obtains an 11-layer network that exceeds several members of the VGG model family on ImageNet, and can train a VGG-11 model to the same accuracy as end-to-end learning. Expand
The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs
TLDR
This paper investigates the supervision vanishing issue in existing backpropagation (BP) methods and proposes to address it via an effective method, called Multi-way BP (MW-BP), which relies on multiple auxiliary losses added to the intermediate layers of the network. Expand
Global-connected network with generalized ReLU activation
TLDR
A novel deep learning architecture aiming to mitigate the gradient-vanishing problem is introduced, in which the earlier hidden layer neurons could be directly connected with the last hidden layer and fed into the softmax layer for classification. Expand
Convolutional networks with cross-layer neurons for image recognition
TLDR
Comparative experiments on several benchmark datasets clearly demonstrate that the proposed model is suitable for training deeper networks and can effectively improve the performance by utilizing cross-layer neurons. Expand
Hyperparameter Optimization for Deep Residual Learning in Image Classification
  • Abbasi Jafar, Myungho Lee
  • Computer Science
  • 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)
  • 2020
TLDR
A manual search approach for ResNet architecture is built by altering the traditional data augmentation proposed by the previous approaches and significantly reduces the computing time compared with the previous automatic approaches. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 49 REFERENCES
ImageNet classification with deep convolutional neural networks
TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective. Expand
Very Deep Convolutional Networks for Large-Scale Image Recognition
TLDR
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. Expand
Going deeper with convolutions
We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual RecognitionExpand
Network In Network
TLDR
With enhanced local modeling via the micro network, the proposed deep network structure NIN is able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers. Expand
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
TLDR
DeCAF, an open-source implementation of deep convolutional activation features, along with all associated network parameters, are released to enable vision researchers to be able to conduct experimentation with deep representations across a range of visual concept learning paradigms. Expand
Deep Learning using Linear Support Vector Machines
  • Y. Tang
  • Computer Science, Mathematics
  • 2013
TLDR
The results using L2-SVMs show that by simply replacing softmax with linear SVMs gives significant gains on popular deep learning datasets MNIST, CIFAR-10, and the ICML 2013 Representation Learning Workshop's face expression recognition challenge. Expand
Visualizing and Understanding Convolutional Networks
TLDR
A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. Expand
Discriminative Transfer Learning with Tree-based Priors
TLDR
This work proposes a method for improving classification performance for high capacity classifiers by discovering similar classes and transferring knowledge among them, which learns to organize the classes into a tree hierarchy, and proposes an algorithm for learning the underlying tree structure. Expand
Understanding Deep Architectures using a Recursive Convolutional Network
TLDR
The notion that adding layers alone increases computational power, within the context of convolutional layers is empirically confirmed and the number of feature maps appears ancillary, and finds most of its benefit through the introduction of more weights. Expand
Traffic sign recognition with multi-scale Convolutional Networks
TLDR
This work applies Convolutional Networks (ConvNets) to the task of traffic sign classification as part of the GTSRB competition, and yields the 2nd-best accuracy above the human performance. Expand
...
1
2
3
4
5
...