Pixel-Level Encoding and Depth Layering for Instance-Level Semantic Labeling

@inproceedings{Uhrig2016PixelLevelEA,
  title={Pixel-Level Encoding and Depth Layering for Instance-Level Semantic Labeling},
  author={Jonas Uhrig and Marius Cordts and Uwe Franke and Thomas Brox},
  booktitle={German Conference on Pattern Recognition},
  year={2016}
}
Recent approaches for instance-aware semantic labeling have augmented convolutional neural networks (CNNs) with complex multi-task architectures or computationally expensive graphical models. We present a method that leverages a fully convolutional network (FCN) to predict semantic labels, depth and an instance-based encoding using each pixel’s direction towards its corresponding instance center. Subsequently, we apply low-level computer vision techniques to generate state-of-the-art instance… 
View PDF on arXiv
159 Citations
Highly Influential Citations
12
Background Citations
74
Methods Citations
62
Results Citations
3

159 Citations

Bridging Category-level and Instance-level Semantic Image Segmentation

An approach to instance-level image segmentation that is built on top of category-level segmentation, which follows a different pipeline to the popular detect-then-segment approaches that first predict instances' bounding boxes.

Deep Watershed Transform for Instance Segmentation

  • Min BaiR. Urtasun
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
This paper presents a simple yet powerful end-to-end convolutional neural network that achieves more than double the performance over the state-of-the-art on the challenging Cityscapes Instance Level Segmentation task.

Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation

A multi-resolution reconstruction architecture based on a Laplacian pyramid that uses skip connections from higher resolution feature maps and multiplicative gating to successively refine segment boundaries reconstructed from lower-resolution maps is described.

Understanding Convolution for Semantic Segmentation

DUC is designed to generate pixel-level prediction, which is able to capture and decode more detailed information that is generally missing in bilinear upsampling, and a hybrid dilated convolution (HDC) framework in the encoding phase is proposed.

Layered Embeddings for Amodal Instance Segmentation

The proposed method extends upon the representational output of semantic instance segmentation by explicitly including both visible and occluded parts, and can accurately estimate complete masks in the presence of occlusion and outperform leading top-down bounding-box approaches.

Proposal-Based Instance Segmentation With Point Supervision

A method called WISE-Net is proposed that only requires point-level annotations for instance segmentation with point- level supervision and obtains competitive results compared to fully-supervised methods in certain scenarios.

Semantic Instance Segmentation via Deep Metric Learning

A new method for semantic instance segmentation is proposed, by first computing how likely two pixels are to belong to the same object, and then by grouping similar pixels together, based on a deep, fully convolutional embedding model.

End-to-End Training of Hybrid CNN-CRF Models for Semantic Segmentation using Structured Learning

This work tackles the problem of semantic image segmentation with a combination of convolutional neural networks (CNNs) and conditional random fields (CRFs) and achieves an intersection over union score of 62.4 in the test set of the cityscapes pixel-level semantic labeling task.

Semantic Instance Segmentation for Autonomous Driving

This work proposes a discriminative loss function, operating at pixel level, that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step.

Learning Semantics-aware Distance Map with Semantics Layering Network for Amodal Instance Segmentation

This work introduces a new representation, namely a semantics-aware distance map (sem-dist map), to serve as a target for amodal segmentation instead of the commonly used masks and heatmaps, and introduces a novel convolutional neural network architecture, which is referred to as semantic layering network, to estimate sem-dist maps layer by layer.
...

References

SHOWING 1-10 OF 41 REFERENCES

Monocular Object Instance Segmentation and Depth Ordering with CNNs

A Markov random field is developed which takes as input the predictions of convolutional neural nets applied at overlapping patches of different resolutions, as well as the output of a connected component algorithm and aims to predict accurate instance-level segmentation and depth ordering.

Instance-Level Segmentation with Deep Densely Connected MRFs

This paper forms the global labeling problem with a novel densely connected Markov random field and shows how to encode various intuitive potentials in a way that is amenable to efficient mean field inference.

Towards unified depth and semantic prediction from a single image

This work proposes a unified framework for joint depth and semantic prediction that effectively leverages the advantages of both tasks and provides the state-of-the-art results.

Instance Segmentation of Indoor Scenes Using a Coverage Loss

This work introduces a model to perform both semantic and instance segmentation simultaneously, and introduces a new higher-order loss function that directly minimizes the coverage metric and evaluate a variety of region features, including those from a convolutional network.

Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation

This work shows how to improve semantic segmentation through the use of contextual information, specifically, ' patch-patch' context between image regions, and 'patch-background' context, and formulate Conditional Random Fields with CNN-based pairwise potential functions to capture semantic correlations between neighboring patches.

Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation

Expectation-Maximization (EM) methods for semantic image segmentation model training under weakly supervised and semi-supervised settings are developed and extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentsation benchmark, while requiring significantly less annotation effort.

Fully convolutional networks for semantic segmentation

The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.

Instance-Aware Semantic Segmentation via Multi-task Network Cascades

  • Jifeng DaiKaiming HeJian Sun
  • Computer Science
    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
This paper presents Multitask Network Cascades for instance-aware semantic segmentation, which consists of three networks, respectively differentiating instances, estimating masks, and categorizing objects, and develops an algorithm for the nontrivial end-to-end training of this causal, cascaded structure.

Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs

This paper forms the global labeling problem with a novel densely connected Markov random field and shows how to encode various intuitive potentials in a way that is amenable to efficient mean field inference.

Proposal-Free Network for Instance-Level Object Segmentation

A Proposal-Free Network (PFN) is proposed to address the instance-level object segmentation problem, which outputs the numbers of instances of different categories and the pixel-level information on i) the coordinates of the instance bounding box each pixel belongs to, and ii) the confidences ofDifferent categories for each pixel, based on pixel-to-pixel deep convolutional neural network.