Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation

  title={Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation},
  author={Lian Xu and Wanli Ouyang and Bennamoun and Farid Boussaid and Ferdous Sohel and Dan Xu},
  journal={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},
  • Lian XuWanli Ouyang Dan Xu
  • Published 25 July 2021
  • Computer Science
  • 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Semantic segmentation is a challenging task in the absence of densely labelled data. Only relying on class activation maps (CAM) with image-level labels provides deficient segmentation supervision. Prior works thus consider pre-trained models to produce coarse saliency maps to guide the generation of pseudo segmentation labels. However, the commonly used off-line heuristic generation process cannot fully exploit the benefits of these coarse saliency maps. Motivated by the significant inter-task… 

Figures and Tables from this paper

Exploiting Shape Cues for Weakly Supervised Semantic Segmentation

Dual Progressive Transformations for Weakly Supervised Semantic Segmentation

Experimental evaluations show that the proposed Convolutional Neural Networks Refined Transformer (CRT) achieves the new state-of-the-art performance on both the weakly supervised semantic segmentation task theWeakly supervised object localization task, which outperform others by a large margin.

TransCAM: Transformer Attention-based CAM Refinement for Weakly Supervised Semantic Segmentation

TransCAM is a Conformer-based solution to WSSS that explicitly leverages the attention weights from the transformer branch of the Conformer to refine the CAM generated from the CNN branch and achieves a new state-of-the-art performance of 69.3% and 69.6% on the respective PASCAL VOC 2012 validation and test sets.

Towards Noiseless Object Contours for Weakly Supervised Semantic Segmentation

A SANCE model is trained which utilizes an auxiliary segmentation module to supplement high-level semantic information for contour training by backbone feature sharing and online label supervision and provides more accurate localization map than CAM for pseudo label generation.

Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Pooling

The second drawback of CAMs is alleviated with the proposed hybrid pooling, which incorporates the global average and local discriminative information to simultaneously ensure object completeness and reduce background regions.

Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentation

. Weakly Supervised Semantic Segmentation (WSSS) research has explored many directions to improve the typical pipeline CNN plus class activation maps (CAM) plus refinements, given the image-class

Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation

Results show the proposed SIPE achieves new state-of-the-art performance using only image-level labels and further optimizes the feature representation and empowers a self-correction ability of prototype exploration.

Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation

A novel framework that derives pixel-level selfsupervision from given image-level supervision that shows state-of-the-art WSSS performance both on the train and validation sets on the PASCAL VOC 2012 dataset is proposed.

GETAM: Gradient-weighted Element-wise Transformer Attention Map for Weakly-supervised Semantic segmentation

The Gradient-weighted Element-wise Transformer Attention Map (GETAM) is introduced and the first transformer-based WSSS approach is proposed, which outperform not only the state-of-the-art end-to-end approaches by a significant margin, but also most of the multi-stage methods.

Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation

This work proposes regional semantic contrast and aggregation (RCA), equipped with a regional memory bank to store massive, diverse object patterns appearing in training data, which acts as strong support for exploration of dataset-level semantic structure.



Learning Saliency-Free Model with Generic Features for Weakly-Supervised Semantic Segmentation

Experiments show that the proposed saliency-free method outperforms the previous approaches under the same weakly-supervised setting and achieves superior segmentation results, which are 64.5% on the validation set and 64.6%" on the test set concerning mIoU metric.

Learning Pixel-Level Semantic Affinity with Image-Level Supervision for Weakly Supervised Semantic Segmentation

  • Jiwoon AhnSuha Kwak
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
On the PASCAL VOC 2012 dataset, a DNN learned with segmentation labels generated by the method outperforms previous models trained with the same level of supervision, and is even as competitive as those relying on stronger supervision.

Weakly-Supervised Semantic Segmentation via Sub-Category Exploration

This work proposes a simple yet effective approach that introduces a self-supervised task by exploiting the sub- category information and performs clustering on image features to generate pseudo sub-categories labels within each annotated parent class.

Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation

A unified multi-task learning framework to jointly solve WSSS and SD using a single network, i.e. saliency and segmentation network (SSNet), which compares favorably against state-of-the-art weakly supervised segmentation methods and fully supervised saliency detection methods.

Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning

An iterative algorithm to learn reliable pixel affinity without accurate annotation is proposed, which consists of two branches, a unary segmentation network which learns the label probabilities for each pixel, and a pairwise affinity network which learning affinity matrix and refines the probability map generated from the unary network.

Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach

This work harnesses the image-level labels to produce reliable pixel-level annotations and design a fully end-to-end network to learn to predict segmentation maps and gets a new state-of-the-art performance on the Pascal VOC.

Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation

A self-supervised equivariant attention mechanism (SEAM) to discover additional supervision and narrow the gap between full and weak supervisions, and a pixel correlation module (PCM), which exploits context appearance information and refines the prediction of current pixel by its similar neighbors, leading to further improvement on CAMs consistency.

Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features

An iterative bottom-up and top-down framework which alternatively expands object regions and optimizes segmentation network and outperforms previous state-of-the-art methods by a large margin is proposed.

Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation

This work designs a graph neural network (GNN) for group-wise semantic segmentation (WSSS), wherein input images are represented as graph nodes, and the underlying relations between a pair of images are char- acterized by an efficient co-attention mechanism.

Discovering Class-Specific Pixels for Weakly-Supervised Semantic Segmentation

It is shown that properly combining saliency and attention maps allows for reliable cues capable of significantly boosting the performance, and a simple yet powerful hierarchical approach to discover the class-agnostic salient regions, obtained using a salient object detector, is proposed.