PreNet: Parallel Recurrent Neural Networks for Image Classification

  title={PreNet: Parallel Recurrent Neural Networks for Image Classification},
  author={Junbo Wang and Wei Wang and Liang Wang and Tieniu Tan},
Convolutional Neural Networks (CNNs) have made outstanding achievements in computer vision, e.g., image classification and object detection, by modelling the receptive field of visual cortex with convolution and pooling operations. However, CNNs have ignored to model the long-range spatial contextual information in images. It has long been believed that recurrent neural networks (RNNs) can model temporal sequences well by virtue of horizontal connections, and have been successfully applied in… 
The deep-feedback deraining network based on the self-attention mechanism
This paper proposes the deraining network via the deep feedback based on the self-attention mechanism, the network uses feedback connections to feedback high-order information to low- order information and corrects the low-orderInformation, and proves that the network has strong reconstruction ability, and can generate clear and high-quality deraining images.


ImageNet classification with deep convolutional neural networks
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
Network In Network
With enhanced local modeling via the micro network, the proposed deep network structure NIN is able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers.
Convolutional Kernel Networks
This paper proposes a new type of convolutional neural network (CNN) whose invariance is encoded by a reproducing kernel, and bridges a gap between the neural network literature and kernels, which are natural tools to model invariance.
Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
This work re-arranges the traditional cuboid order of computations in MD-LSTM in pyramidal fashion and achieves best known pixel-wise brain image segmentation results on MRBrainS13 (and competitive results on EM-ISBI12).
Spatially-sparse convolutional neural networks
A CNN for processing spatially-sparse inputs, motivated by the problem of online handwriting recognition, and applying a deep convolutional network using sparsity has resulted in a substantial reduction in test error on the CIFAR small picture datasets.
Fully Convolutional Networks for Semantic Segmentation
It is shown that convolutional networks by themselves, trained end- to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation.
ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
The proposed network, called ReNet, replaces the ubiquitous convolution+pooling layer of the deep convolutional neural network with four recurrent neural networks that sweep horizontally and vertically in both directions across the image.
Speech recognition with deep recurrent neural networks
This paper investigates deep recurrent neural networks, which combine the multiple levels of representation that have proved so effective in deep networks with the flexible use of long range context that empowers RNNs.
Deeply-Supervised Nets
The proposed deeply-supervised nets (DSN) method simultaneously minimizes classification error while making the learning process of hidden layers direct and transparent, and extends techniques from stochastic gradient methods to analyze the algorithm.
Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks
This paper introduces a globally trained offline handwriting recogniser that takes raw pixel data as input and does not require any alphabet specific preprocessing, and can therefore be used unchanged for any language.