Attention to Scale: Scale-Aware Semantic Image Segmentation

@article{Chen2016AttentionTS,
  title={Attention to Scale: Scale-Aware Semantic Image Segmentation},
  author={Liang-Chieh Chen and Yi Yang and Jiang Wang and Wei Xu and Alan Loddon Yuille},
  journal={2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2016},
  pages={3640-3649}
}
Incorporating multi-scale features in fully convolutional neural networks (FCNs) has been a key element to achieving state-of-the-art performance on semantic image segmentation. [...] Key Method We adapt a state-of-the-art semantic image segmentation model, which we jointly train with multi-scale input images and the attention model. The proposed attention model not only outperforms averageand max-pooling, but allows us to diagnostically visualize the importance of features at different positions and scales…Expand
Semantic Image Segmentation Based on Attentions to Intra Scales and Inner Channels
TLDR
A new attention mechanism that not only learns weights between different scales but also allocates importance to subregions of inner channels to enhance impact of each feature map in intermediate layers on performance. Expand
GSTO: Gated Scale-Transfer Operation for Multi-Scale Feature Learning in Semantic Segmentation
TLDR
The Gated Scale-Transfer Operation (GSTO) is proposed to properly transit spatial-filtered features to another scale and achieves new state-of-the-art results on multiple benchmarks for semantic segmentation including Cityscapes, LIP, and Pascal Context with a negligible extra computational cost. Expand
Scale-aware spatial pyramid pooling with both encoder-mask and scale-attention for semantic segmentation
TLDR
Three modules are proposed: Scale-aware spatial pyramid pool module, Encoder mask module and Scale-attention module (SSPP-ES) that solve the challenges of semantic segmentation of scenes by capturing the appropriate scale, rich detail and contextual dependencies in a feature representation. Expand
Attention to Refine through Multi-Scales for Semantic Segmentation
TLDR
A novel attention model for semantic segmentation, which aggregates multi-scale and context features to refine prediction, achieves quite competitive results on PASCAL VOC 2012 and ADE20K datasets, which surpass baseline and related works. Expand
Multi-Receptive Atrous Convolutional Network for Semantic Segmentation
TLDR
This paper adapts the ResNet-101 model as the backbone network and proposes a MRACN segmentation model (MRACN-Seg), which captures the multi-receptive features and the global features at different receptive scales of the input. Expand
Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images
TLDR
This work presents a novel embedded single stream optimization method based on the MR model to avoid approximations without sacrificing expressive power and proposes a novel network that combines the dilated, multi-scale strategies with the single stream MR optimization method in the deep learning architecture to further improve the performance. Expand
Multiscale Attention-Based Prototypical Network For Few-Shot Semantic Segmentation
TLDR
This work proposes a novel prototypical network (MAPnet) with multiscale feature attention that adaptively integrate multiple similarity-guided probability maps by attention mechanism, yielding an optimal pixel-wise prediction. Expand
Fully Convolutional Neural Networks with Full-Scale-Features for Semantic Segmentation
TLDR
This work proposes a novel method to involve full-scale-features into the fully convolutional neural networks (FCNs) for Semantic Segmentation, and achieves a state-of-art result. Expand
Scale-Aware Feature Network for Weakly Supervised Semantic Segmentation
TLDR
The proposed SAFN uses an attention module to learn the relative weights of multi-scale features in a modified fully convolutional network with dilated convolutions, which leads to efficient enlargements of the receptive fields of view and produces dense object localization maps. Expand
Hierarchical Multi-Scale Attention for Semantic Segmentation
TLDR
This work presents an attention-based approach to combining multi-scale predictions, and shows that predictions at certain scales are better at resolving particular failures modes, and that the network learns to favor those scales for such cases in order to generate better predictions. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 66 REFERENCES
The application of two-level attention models in deep convolutional neural network for fine-grained image classification
TLDR
This paper proposes to apply visual attention to fine-grained classification task using deep neural network and achieves the best accuracy under the weakest supervision condition, and is competitive against other methods that rely on additional annotations. Expand
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
TLDR
This work shows how to improve semantic segmentation through the use of contextual information, specifically, ' patch-patch' context between image regions, and 'patch-background' context, and formulate Conditional Random Fields with CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Expand
Fully Convolutional Networks for Semantic Segmentation
TLDR
It is shown that convolutional networks by themselves, trained end- to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation. Expand
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
TLDR
This paper proposes a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Expand
Recurrent Convolutional Neural Networks for Scene Labeling
TLDR
This work proposes an approach that consists of a recurrent convolutional neural network which allows us to consider a large input context while limiting the capacity of the model, and yields state-of-the-art performance on both the Stanford Background Dataset and the SIFT FlowDataset while remaining very fast at test time. Expand
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
TLDR
This work brings together methods from DCNNs and probabilistic graphical models for addressing the task of pixel-level classification by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF). Expand
Joint Object and Part Segmentation Using Deep Learned Potentials
TLDR
This paper proposes a joint solution that tackles semantic object and part segmentation simultaneously, in which higher object-level context is provided to guide part segmentsation, and more detailed part-level localization is utilized to refine object segmentation. Expand
Semantic Image Segmentation via Deep Parsing Network
TLDR
This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Expand
Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
TLDR
Expectation-Maximization (EM) methods for semantic image segmentation model training under weakly supervised and semi-supervised settings are developed and extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentsation benchmark, while requiring significantly less annotation effort. Expand
Learning Deconvolution Network for Semantic Segmentation
TLDR
A novel semantic segmentation algorithm by learning a deep deconvolution network on top of the convolutional layers adopted from VGG 16-layer net, which demonstrates outstanding performance in PASCAL VOC 2012 dataset. Expand
...
1
2
3
4
5
...