• Corpus ID: 16364905

End-to-End Instance Segmentation and Counting with Recurrent Attention

@article{Ren2016EndtoEndIS,
  title={End-to-End Instance Segmentation and Counting with Recurrent Attention},
  author={Mengye Ren and Richard S. Zemel},
  journal={ArXiv},
  year={2016},
  volume={abs/1605.09410}
}
While convolutional neural networks have gained impressive success recently in solving structured prediction problems such as semantic segmentation, it remains a challenge to differentiate individual object instances in the scene. [] Key Method Techniques that combine large graphical models with low-level vision have been proposed to address this problem; however, we propose an end-to-end recurrent neural network (RNN) architecture with an attention mechanism to model a human-like counting process, and…
View PDF on arXiv
61 Citations
Highly Influential Citations
4
Background Citations
29
Methods Citations
19
Results Citations
3

Figures and Tables from this paper

61 Citations

Semantic Instance Segmentation for Autonomous Driving

This work proposes a discriminative loss function, operating at pixel level, that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step.

Semantic Instance Segmentation for Autonomous Driving Bert

This work proposes to tackleantic instance segmentation with a discriminative loss function, operating at pixel level, that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step.

Deep Variational Instance Segmentation

A novel algorithm that directly utilizes a fully convolutional network (FCN) to predict instance labels and extends the classical Mumford-Shah variational segmentation problem to be able to handle permutation-invariant labels in the ground truth of instance segmentation.

Deep Watershed Transform for Instance Segmentation

  • Min BaiR. Urtasun
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
This paper presents a simple yet powerful end-to-end convolutional neural network that achieves more than double the performance over the state-of-the-art on the challenging Cityscapes Instance Level Segmentation task.

Semantic Instance Segmentation with a Discriminative Loss Function

This work proposes an approach of combining an off-the-shelf network with a principled loss function inspired by a metric learning objective that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step.

Boundary-Aware Instance Segmentation

This paper introduces a novel object segment representation based on the distance transform of the object masks, and designs an object mask network (OMN) with a new residual-deconvolution architecture that infers such a representation and decodes it into the final binary object mask.

Fast Instance and Semantic Segmentation Exploiting Local Connectivity, Metric Learning, and One-Shot Detection for Robotics

This paper addresses the problem of jointly performing semantic segmentation as well as instance segmentation in an online fashion, so that autonomous robots can use this information on-the-go and without sacrificing accuracy.

Multiscale Attention-Based Prototypical Network For Few-Shot Semantic Segmentation

This work proposes a novel prototypical network (MAPnet) with multiscale feature attention that adaptively integrate multiple similarity-guided probability maps by attention mechanism, yielding an optimal pixel-wise prediction.

InstanceCut: From Edges to Instances with MultiCut

To reason globally about the optimal partitioning of an image into instances, the authors combine these two modalities into a novel MultiCut formulation, which achieves the best result among all published methods, and performs particularly well for rare object classes.

SGN: Sequential Grouping Networks for Instance Segmentation

This paper proposes Sequential Grouping Networks, a sequence of neural networks, each solving a sub-grouping problem of increasing semantic complexity in order to gradually compose objects out of pixels to tackle the problem of object instance segmentation.
...

30 References

Recurrent Instance Segmentation

This work proposes a new instance segmentation paradigm consisting in an end-to-end method that learns how to segment instances sequentially, based on a recurrent neural network that sequentially finds objects and their segmentations one at a time.

Learning to decompose for object detection and instance segmentation

This work proposes a novel end-to-end trainable deep neural network architecture that generates the correct number of object instances and their bounding boxes (or segmentation masks) given an image, using only a single network evaluation without any pre- or post-processing steps.

Instance-Aware Semantic Segmentation via Multi-task Network Cascades

  • Jifeng DaiKaiming HeJian Sun
  • Computer Science
    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
This paper presents Multitask Network Cascades for instance-aware semantic segmentation, which consists of three networks, respectively differentiating instances, estimating masks, and categorizing objects, and develops an algorithm for the nontrivial end-to-end training of this causal, cascaded structure.

Proposal-Free Network for Instance-Level Object Segmentation

A Proposal-Free Network (PFN) is proposed to address the instance-level object segmentation problem, which outputs the numbers of instances of different categories and the pixel-level information on i) the coordinates of the instance bounding box each pixel belongs to, and ii) the confidences ofDifferent categories for each pixel, based on pixel-to-pixel deep convolutional neural network.

Fully convolutional networks for semantic segmentation

The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.

Monocular Object Instance Segmentation and Depth Ordering with CNNs

A Markov random field is developed which takes as input the predictions of convolutional neural nets applied at overlapping patches of different resolutions, as well as the output of a connected component algorithm and aims to predict accurate instance-level segmentation and depth ordering.

Learning Deconvolution Network for Semantic Segmentation

A novel semantic segmentation algorithm by learning a deep deconvolution network on top of the convolutional layers adopted from VGG 16-layer net, which demonstrates outstanding performance in PASCAL VOC 2012 dataset.

Pixel-Level Encoding and Depth Layering for Instance-Level Semantic Labeling

This work presents a method that leverages a fully convolutional network (FCN) to predict semantic labels, depth and an instance-based encoding using each pixel’s direction towards its corresponding instance center.

Instance-Level Segmentation with Deep Densely Connected MRFs

This paper forms the global labeling problem with a novel densely connected Markov random field and shows how to encode various intuitive potentials in a way that is amenable to efficient mean field inference.

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

This work introduces a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals and further merge RPN and Fast R-CNN into a single network by sharing their convolutionAL features.