Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning

  title={Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning},
  author={Alexander Vezhnevets and Joachim M. Buhmann},
  journal={2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition},
  • A. Vezhnevets, J. Buhmann
  • Published 13 June 2010
  • Computer Science
  • 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
We address the task of learning a semantic segmentation from weakly supervised data. Our aim is to devise a system that predicts an object label for each pixel by making use of only image level labels during training – the information whether a certain object is present or not in the image. Such coarse tagging of images is faster and easier to obtain as opposed to the tedious task of pixelwise labeling required in state of the art systems. We cast this task naturally as a multiple instance… 

Figures and Tables from this paper

Image Piece Learning for Weakly Supervised Semantic Segmentation

A novel conditional random field (CRF) based framework for weakly supervised semantic segmentation that outperforms or is comparable to state-of-the-art segmentation methods and is fit for domain adaption.

Weakly Supervised Semantic Segmentation with Convolutional Networks

A Convolutional Neural Network-based model is proposed, which is constrained during training to put more weight on pixels which are important for classifying the image, and which beats the state of the art results in weakly supervised object segmentation task by a large margin.

Weakly supervised semantic segmentation with a multi-image model

A novel method for weakly supervised semantic segmentation using a multi-image model (MIM) - a graphical model for recovering the pixel labels of the training images and introducing an “objectness” potential, that helps separating objects from background classes.

Active learning for semantic segmentation with expected change

A novel active learning method specifically suited for semantic segmentation of pixels in an image according to the semantic class it belongs to, model the problem as a pairwise CRF and cast active learning as finding its most informative nodes.

A Pixel-Level Meta-Learner for Weakly Supervised Few-Shot Semantic Segmentation

A novel meta-learning framework, which predicts pseudo pixel-level segmentation masks from a limited amount of data and their semantic labels, yet performs favorably against state-of-the-art methods under weakly supervised settings.

Weakly Supervised RBM for Semantic Segmentation

A weakly supervised Restricted Boltzmann Machines (WRBM) approach to deal with the task of semantic segmentation with only image-level labels available, which deals with the problems of label imbalance and diverse backgrounds by adapting the block size to the label frequency and appending hidden response blocks corresponding to backgrounds respectively.

A Probabilistic Associative Model for Segmenting Weakly Supervised Images

This paper proposes a new weakly supervised image segmentation model, focusing on learning the semantic associations between superpixel sets (graphlets in this paper), and presents a hierarchical Bayesian network to capture the semantic association between postembedding graphlets.

Simultaneous Semantic Segmentation of a Set of Partially Labeled Images

A new approach to semantic segmentation of a set of partially-labelled images, using a formulation considering information from multiple visual similar images is proposed, demonstrating evident performance improvement of the proposed approach.

Simultaneous semantic segmentation of a set of partially labeled images

  • Q. TianBaoxin Li
  • Computer Science
    2016 IEEE Winter Conference on Applications of Computer Vision (WACV)
  • 2016
A new approach to semantic segmentation of a set of partially-labelled images, using a formulation considering information from multiple visual similar images is proposed, demonstrating evident performance improvement of the proposed approach.

Weakly Supervised Learning with Deep Convolutional Neural Networks for Semantic Segmentation: Understanding Semantic Layout of Images with Minimum Human Supervision

This article provides a comprehensive overview of weakly supervised approaches for semantic segmentation, describing how the approaches overcome the limitations and discuss research directions worthy of investigation to improve performance.



Weakly Supervised Object Localization with Stable Segmentations

This paper introduces the challenging Landmarks-18 dataset, a collection of photographs of famous landmarks from around the world, and proposes a new object recognition system called MILSS, which shows highly competitive object categorization results on the Caltech dataset.

Joint multi-label multi-instance learning for image classification

This work proposes an integrated multi- label multi-instance learning (MLMIL) approach based on hidden conditional random fields (HCRFs), which simultaneously captures both the connections between semantic labels and regions, and the correlations among the labels in a single formulation.

Robust Higher Order Potentials for Enforcing Label Consistency

This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner based on higher order conditional random fields and uses potentials defined on sets of pixels generated using unsupervised segmentation algorithms.

Semi-Supervised Random Forests

This work develops a novel multi-class margin definition for the unlabeled data, and proposes a control mechanism based on the out-of-bag error, which prevents the algorithm from degradation if the unl labeled data is not useful for the task.

Semantic texton forests for image categorization and segmentation

The proposed semantic texton forests are ensembles of decision trees that act directly on image pixels, and therefore do not need the expensive computation of filter-bank responses or local descriptors, and give at least a five-fold increase in execution speed.

Multi-Instance Multi-Label Learning with Application to Scene Classification

This paper formalizes multi-instance multi-label learning, where each training example is associated with not only multiple instances but also multiple class labels, and proposes the MIMLBOOST and MIMLSVM algorithms which achieve good performance in an application to scene classification.

TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation

A new approach to learning a discriminative model of object classes, incorporating appearance, shape and context information efficiently, is proposed, which is used for automatic visual recognition and semantic segmentation of photographs.

Multi-Task Feature Learning

The method builds upon the well-known 1-norm regularization problem using a new regularizer which controls the number of learned features common for all the tasks, and develops an iterative algorithm for solving it.

Multiscale conditional random fields for image labeling

An approach to include contextual features for labeling images, in which each pixel is assigned to one of a finite set of labels, are incorporated into a probabilistic framework, which combines the outputs of several components.

Region Classification with Markov Field Aspect Models

  • J. VerbeekB. Triggs
  • Computer Science
    2007 IEEE Conference on Computer Vision and Pattern Recognition
  • 2007
Combining spatial and aspect models significantly improves the region-level classification accuracy, and models trained with image-level labels outperform PLSA trained with pixel-level ones.