From image-level to pixel-level labeling with Convolutional Networks

@article{Pinheiro2015FromIT,
  title={From image-level to pixel-level labeling with Convolutional Networks},
  author={Pedro H. O. Pinheiro and Ronan Collobert},
  journal={2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2015},
  pages={1713-1721}
}
We are interested in inferring object segmentation by leveraging only object class information, and by considering only minimal priors on the object segmentation task. [...] Key Method We propose a Convolutional Neural Network-based model, which is constrained during training to put more weight on pixels which are important for classifying the image.Expand
Learning to segment with image-level annotations
TLDR
This paper explores a more challenging problem by learning to segment under image-level annotations by incorporating the hypotheses-aware classification and cross-image contextual refinement in a supervised manner, and proposes two network training strategies for achieving good segmentation performance. Expand
Exploiting Saliency for Object Segmentation from Image Level Labels
TLDR
This paper proposes using a saliency model as additional information and hereby exploit prior knowledge on the object extent and image statistics and shows how to combine both information sources in order to recover 80% of the fully supervised performance of pixel-wise semantic labelling. Expand
Large-Scale Image Segmentation with Convolutional Networks
TLDR
This thesis proposes a CNN-based discriminative approach that is able to learn segmentation proposals from raw pixels, and shows that the proposals generated by this approach, when coupled with a standard state-of-the-art object detection pipeline, achieve considerably better performance than previous proposals methods. Expand
Learning to Segment With Image-Level Supervision
TLDR
This paper proposes a model that generates auxiliary labels for each image, while simultaneously forcing the output of the CNN to satisfy the mean-field constraints imposed by a conditional random field, and achieves the state-of-the-art for weakly supervised semantic image segmentation on VOC 2012 dataset. Expand
Incorporating Network Built-in Priors in Weakly-Supervised Semantic Segmentation
TLDR
This work proposes a novel method to extract accurate masks from networks pre-trained for the task of object recognition, thus forgoing external objectness modules, and shows how foreground/background masks can be obtained from the activations of higher-level convolutional layers of a network. Expand
Instance Segmentation with Point Supervision
TLDR
The method obtains competitive results compared to fully-supervised methods in certain scenarios; outperforms fully- and weakly- supervised methods with a fixed annotation budget; and is a first strong baseline for instance segmentation with point-level supervision. Expand
Learning to Exploit the Prior Network Knowledge for Weakly Supervised Semantic Segmentation
TLDR
This paper introduces a novel weakly supervised semantic segmentation model which is able to learn from image labels and just image labels, and generates accurate class-specific segmentation masks from these regions, where neither external objectness nor saliency algorithms are required. Expand
Weakly Supervised Conditional Random Fields Model for Semantic Segmentation with Image Patches
TLDR
A weakly supervised ISS approach is proposed, in which the challenging problem of label inference from image-level to pixel-level will be particularly addressed, using image patches and conditional random fields (CRF). Expand
Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation.
TLDR
This paper proposes a multiple instance learning (MIL) framework, which can be trained in an end-to-end manner using training images with image-level labels and achieves state-of-the-art performance for both weakly supervised instance segmentation and semantic segmentation. Expand
Where are the Masks: Instance Segmentation with Image-level Supervision
TLDR
A novel framework that can effectively train with image-level labels, which are significantly cheaper to acquire, and achieves new state-of-the-art results for this problem setup is proposed. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 33 REFERENCES
Weakly supervised semantic segmentation with a multi-image model
TLDR
A novel method for weakly supervised semantic segmentation using a multi-image model (MIM) - a graphical model for recovering the pixel labels of the training images and introducing an “objectness” potential, that helps separating objects from background classes. Expand
Representative Discovery of Structure Cues for Weakly-Supervised Image Segmentation
TLDR
Experimental results show that the proposed approach outperforms state-of-the-art weakly-supervised image segmentation methods, on five popular segmentation data sets and performs competitively to the fully- supervised segmentation models. Expand
Recurrent Convolutional Neural Networks for Scene Labeling
TLDR
This work proposes an approach that consists of a recurrent convolutional neural network which allows us to consider a large input context while limiting the capacity of the model, and yields state-of-the-art performance on both the Stanford Background Dataset and the SIFT FlowDataset while remaining very fast at test time. Expand
Learning Hierarchical Features for Scene Labeling
TLDR
A method that uses a multiscale convolutional network trained from raw pixels to extract dense feature vectors that encode regions of multiple sizes centered on each pixel, alleviates the need for engineered features, and produces a powerful representation that captures texture, shape, and contextual information. Expand
Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning
  • A. Vezhnevets, J. Buhmann
  • Computer Science
  • 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • 2010
TLDR
An external task of geometric context estimation is used to improve on the task of semantic segmentation, using Semantic Texton Forest (STF) as the basic framework and extending it for the MIL setting. Expand
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
TLDR
This paper proposes a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Expand
Simultaneous Detection and Segmentation
TLDR
This work builds on recent work that uses convolutional neural networks to classify category-independent region proposals (R-CNN), introducing a novel architecture tailored for SDS, and uses category-specific, top-down figure-ground predictions to refine the bottom-up proposals. Expand
Weakly supervised structured output learning for semantic segmentation
TLDR
A parametric family of structured models is defined, were each model weights visual cues in a different way and a Maximum Expected Agreement model selection principle is proposed that evaluates the quality of a model from the family without looking at superpixel labels. Expand
Probabilistic Graphlet Cut: Exploiting Spatial Structure Cue for Weakly Supervised Image Segmentation
TLDR
A novel image segmentation algorithm is proposed, called graph let cut, that leverages the learned graph let distribution in measuring the homogeneity of a set of spatially structured super pixels. Expand
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
TLDR
This work designs a method to reuse layers trained on the ImageNet dataset to compute mid-level image representation for images in the PASCAL VOC dataset, and shows that despite differences in image statistics and tasks in the two datasets, the transferred representation leads to significantly improved results for object and action classification. Expand
...
1
2
3
4
...