Densely Connected Convolutional Networks

@article{Huang2017DenselyCC,
  title={Densely Connected Convolutional Networks},
  author={Gao Huang and Zhuang Liu and Kilian Q. Weinberger},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2017},
  pages={2261-2269}
}
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. [] Key Method For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation…

Figures and Tables from this paper

Sparsely Connected Convolutional Networks
TLDR
This work analyzes the feature aggregation patterns of ResNets and DenseNets under a uniform aggregation view framework and shows that both structures densely gather features from previous layers in the network but combine them in their respective ways: summation or concatenation.
Do We Need Fully Connected Output Layers in Convolutional Networks?
TLDR
It is demonstrated that fixed classifiers offer no additional benefit compared to simply removing the output layer along with its parameters, and that the typical approach of having a fully connected final output layer is inefficient in terms of parameter count.
Aggregated squeeze-and-excitation transformations for densely connected convolutional networks
TLDR
This paper proposes a lightweight Densely Connected and Inter-Sparse Convolutional Networks with aggregated Squeeze-and-Excitation transformations (DenisNet-SE), which achieves better performance than the state-of-the-art networks while requiring fewer parameters.
A Fully Sequential Methodology for Convolutional Neural Networks
TLDR
A fully sequential methodology to construct and train extremely deep convolutional neural networks is proposed and encourages long term dependency to be established among each module of the network, which increases the effective depth of networks with skip connections, as well alleviates multiple optimization difficulties for deep networks.
ADNet: Adaptively Dense Convolutional Neural Networks
TLDR
This paper presents a layer attention based Adaptively Dense Network (ADNet) by adaptively determining the reuse status of hierarchical preceding features by adaptingively determiningThe reuse status is adapted to fuse multi-level internal representations in an effective manner.
Reuse of Mid-Level Feature in Deep Convolutional Neural Network
TLDR
A novel end-to-end model called ReuseNet, which utilizes mid-level features and can utilize features of different levels and also strengthen feature propagation, in order to improve the accuracy of downstream tasks.
Dense short connection network for efficient image classification
TLDR
The dense short connection network (DSCNet) for efficient image classification is designed and proposed, which allows hierarchical dense short connections for multi‐scale context information across different feature channels within a single convolutional layer.
Convolutional Neural Networks with Layer Reuse
TLDR
A CNN architecture, Layer Reuse Network (LruNet), where the convolutional layers are used repeatedly without the need of introducing new layers to get a better performance is proposed.
Multipath-DenseNet: A Supervised ensemble architecture of densely connected convolutional networks
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 59 REFERENCES
Striving for Simplicity: The All Convolutional Net
TLDR
It is found that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks.
Network In Network
TLDR
With enhanced local modeling via the micro network, the proposed deep network structure NIN is able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers.
ImageNet classification with deep convolutional neural networks
TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
Fully Convolutional Networks for Semantic Segmentation
TLDR
It is shown that convolutional networks by themselves, trained end- to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation.
Deep Networks with Stochastic Depth
TLDR
Stochastic depth is proposed, a training procedure that enables the seemingly contradictory setup to train short networks and use deep networks at test time and reduces training time substantially and improves the test error significantly on almost all data sets that were used for evaluation.
Wide Residual Networks
TLDR
This paper conducts a detailed experimental study on the architecture of ResNet blocks and proposes a novel architecture where the depth and width of residual networks are decreased and the resulting network structures are called wide residual networks (WRNs), which are far superior over their commonly used thin and very deep counterparts.
Deep Residual Learning for Image Recognition
TLDR
This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
Deeply-Supervised Nets
TLDR
The proposed deeply-supervised nets (DSN) method simultaneously minimizes classification error while making the learning process of hidden layers direct and transparent, and extends techniques from stochastic gradient methods to analyze the algorithm.
Deeply-Fused Nets
TLDR
The central idea of the approach is deep fusion, i.e., combine the intermediate representations of base networks, where the fused output serves as the input of the remaining part of each base network, and perform such combinations deeply over several intermediate representations.
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
TLDR
This work demonstrates that the intermediate activations of pretrained large-scale classification networks preserve almost all the information of input images except a portion of local spatial details, and investigates joint supervised and unsupervised learning in a large- scale setting by augmenting existing neural networks with decoding pathways for reconstruction.
...
1
2
3
4
5
...