• Corpus ID: 12998557

Striving for Simplicity: The All Convolutional Net

@article{Springenberg2015StrivingFS,
  title={Striving for Simplicity: The All Convolutional Net},
  author={Jost Tobias Springenberg and Alexey Dosovitskiy and Thomas Brox and Martin A. Riedmiller},
  journal={CoRR},
  year={2015},
  volume={abs/1412.6806}
}
Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. [...] Key Method Following this finding -- and building on other recent work for finding simple network structures -- we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10…Expand
Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units
TLDR
This paper proposes a novel, simple yet effective activation scheme called concatenated ReLU (CRelu) and theoretically analyze its reconstruction property in CNNs and integrates CRelu into several state-of-the-art CNN architectures and demonstrates improvement in their recognition performance on CIFAR-10/100 and ImageNet datasets with fewer trainable parameters.
Convolutional Residual Memory Networks
TLDR
This work proposes and evaluates a memory mechanism enhanced convolutional neural network architecture based on augmenting Convolutional residual networks with a long short term memory mechanism, which can yield state of the art performance on the CIFAR-100 benchmark and compares well with other state ofThe art techniques on the SOTA and SVHN benchmarks.
Deep Convolutional Networks without Learning the Classifier Layer
Deep convolutional neural networks (CNNs) are effective and popularly used in a wide variety of computer vision tasks, especially in image classification. Conventionally, they consist of a series of
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
TLDR
The results on the MNIST, CIFAR-10/100 and ImageNet datasets using the NiN, Overfeat, All-CNN and ResNet networks indicate that the proposed Parametric ELU (PELU) has better performances than the non-parametricELU.
Improved Very Deep Recurrent Convolutional Neural Network for Object Recognition
TLDR
A very deep recurrent convolutional neural network approach for object recognition reinforced by integrating recurrent connections to the Convolutional layers and the spatial pyramid pooling is proposed.
Improved Very Deep Recurrent Convolutional Neural Network for Object Recognition
TLDR
A very deep recurrent convolutional neural network approach for object recognition reinforced by integrating recurrent connections to the Convolutional layers and the spatial pyramid pooling is proposed.
Gradually Updated Neural Networks for Large-Scale Image Recognition
TLDR
An alternative method to increase the depth of neural networks by introducing computation orderings to the channels within convolutional layers or blocks, based on which the outputs are gradually computed in a channel-wise manner.
Scale-Invariant Recognition by Weight-Shared CNNs in Parallel
TLDR
A novel network architecture, the weight-shared multi-stage network (WSMS-Net), and focuses on acquiring the scale invariance by constructing of multiple stages of CNNs, which achieves higher classification accuracy on CIFAR-10, CIFar-100 and ImageNet datasets.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 31 REFERENCES
Very Deep Convolutional Networks for Large-Scale Image Recognition
TLDR
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
ImageNet classification with deep convolutional neural networks
TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
Network In Network
TLDR
With enhanced local modeling via the micro network, the proposed deep network structure NIN is able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers.
Deeply-Supervised Nets
TLDR
The proposed deeply-supervised nets (DSN) method simultaneously minimizes classification error while making the learning process of hidden layers direct and transparent, and extends techniques from stochastic gradient methods to analyze the algorithm.
Going deeper with convolutions
We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition
Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
TLDR
It is shown that multilayer perceptrons (MLP) consisting of the L p units achieve the state-of-the-art results on a number of benchmark datasets and the proposed L p unit is evaluated on the recently proposed deep recurrent neural networks (RNN).
Fractional Max-Pooling
TLDR
The form of fractional max-pooling formulated is found to reduce overfitting on a variety of datasets: for instance, it improves on the state of the art for CIFAR-100 without even using dropout.
Visualizing and Understanding Convolutional Networks
TLDR
A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark.
High-Performance Neural Networks for Visual Object Classification
We present a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants. Our feature extractors are neither carefully designed nor pre-wired, but rather learned in a
Learning Multiple Layers of Features from Tiny Images
TLDR
It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network.
...
1
2
3
4
...