What's the Point: Semantic Segmentation with Point Supervision

@article{Bearman2016WhatsTP,
  title={What's the Point: Semantic Segmentation with Point Supervision},
  author={Amy L. Bearman and Olga Russakovsky and Vittorio Ferrari and Li Fei-Fei},
  journal={ArXiv},
  year={2016},
  volume={abs/1506.02106}
}
The semantic image segmentation task presents a trade-off between test time accuracy and training time annotation cost. Detailed per-pixel annotations enable training accurate models but are very time-consuming to obtain; image-level class labels are an order of magnitude cheaper but result in less accurate models. We take a natural step from image-level annotation towards stronger supervision: we ask annotators to point to an object if one exists. We incorporate this point supervision along… 

Proposal-Based Instance Segmentation With Point Supervision

A method called WISE-Net is proposed that only requires point-level annotations for instance segmentation with point- level supervision and obtains competitive results compared to fully-supervised methods in certain scenarios.

PCAMs: Weakly Supervised Semantic Segmentation Using Point Supervision

This paper presents a novel procedure for producing semantic segmentation from images given some point level annotations, which includes point annotations in the training of a convolutional neural network for producing improved localization and class activation maps and training a CNN that is normally fully supervised using pseudo labels in place of ground truth labels.

Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach

This work harnesses the image-level labels to produce reliable pixel-level annotations and design a fully end-to-end network to learn to predict segmentation maps and gets a new state-of-the-art performance on the Pascal VOC.

Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation

This work proposes a novel method to extract markedly more accurate masks from the pre-trained network itself, forgoing external objectness modules, and introduces a new form of inexpensive weak supervision yielding an additional accuracy boost.

Where are the Masks: Instance Segmentation with Image-level Supervision

A novel framework that can effectively train with image-level labels, which are significantly cheaper to acquire, and achieves new state-of-the-art results for this problem setup is proposed.

Pointly-Supervised Instance Segmentation

The existing instance segmentation models developed for full mask supervision can be seamlessly trained with point-based supervision collected via the proposed point annotation scheme, which is approximately 5 times faster than annotating full object masks, making high-quality instance segmentations more accessible in practice.

Incorporating Network Built-in Priors in Weakly-Supervised Semantic Segmentation

This work proposes a novel method to extract accurate masks from networks pre-trained for the task of object recognition, thus forgoing external objectness modules, and shows how foreground/background masks can be obtained from the activations of higher-level convolutional layers of a network.

An Improved Approach to Weakly Supervised Semantic Segmentation

This work relies on the interpretable ability of deep neural networks to generate attention maps with class localization information by back-propagating gradients and employs an off-the-shelf object saliency detector with an iterative erasing strategy to obtain saliency maps with spatial extent information of objects.

All you need are a few pixels: semantic segmentation with PixelPick

This work shows that in order to achieve a good level of segmentation performance, all you need are a few well-chosen pixel labels and demonstrates that deep neural networks can use a handful of such labels to good effect.

DASNet: Reducing Pixel-level Annotations for Instance and Semantic Segmentation

This work aims to achieve high quality instance and semantic segmentation results over a small set of pixel-level mask annotations and a large set of box annotations and demonstrates substantially improved performance compared to existing semi-supervised approaches on PASCAL VOC 2012 dataset.
...

References

SHOWING 1-10 OF 50 REFERENCES

BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation

  • Jifeng DaiKaiming HeJian Sun
  • Computer Science
    2015 IEEE International Conference on Computer Vision (ICCV)
  • 2015
This paper proposes a method that achieves competitive accuracy but only requires easily obtained bounding box annotations, and yields state-of-the-art results on PASCAL VOC 2012 and PASCal-CONTEXT.

From image-level to pixel-level labeling with Convolutional Networks

A Convolutional Neural Network-based model is proposed, which is constrained during training to put more weight on pixels which are important for classifying the image, and which beats the state of the art results in weakly supervised object segmentation task by a large margin.

Learning to segment under various forms of weak supervision

This work proposes a unified approach that incorporates various forms of weak supervision - image level tags, bounding boxes, and partial labels - to produce a pixel-wise labeling on the challenging Siftflow dataset.

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

This paper proposes a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%.

Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation

Expectation-Maximization (EM) methods for semantic image segmentation model training under weakly supervised and semi-supervised settings are developed and extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentsation benchmark, while requiring significantly less annotation effort.

Fully Convolutional Multi-Class Multiple Instance Learning

This work proposes a novel MIL formulation of multi-class semantic segmentation learning by a fully convolutional network that exploits the further supervision given by images with multiple labels.

Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation

Expectation-Maximization (EM) methods for semantic image segmentation model training under weakly supervised and semi-supervised settings are developed and extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentsation benchmark, while requiring significantly less annotation effort.

Simultaneous Detection and Segmentation

This work builds on recent work that uses convolutional neural networks to classify category-independent region proposals (R-CNN), introducing a novel architecture tailored for SDS, and uses category-specific, top-down figure-ground predictions to refine the bottom-up proposals.

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

This work brings together methods from DCNNs and probabilistic graphical models for addressing the task of pixel-level classification by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF).

Weakly supervised semantic segmentation with a multi-image model

A novel method for weakly supervised semantic segmentation using a multi-image model (MIM) - a graphical model for recovering the pixel labels of the training images and introducing an “objectness” potential, that helps separating objects from background classes.