Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation

  title={Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation},
  author={Jungbeom Lee and Eunji Kim and Sungroh Yoon},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
Weakly supervised semantic segmentation produces a pixel-level localization from class labels; but a classifier trained on such labels is likely to restrict its focus to a small discriminative region of the target object. AdvCAM is an attribution map of an image that is manipulated to increase the classification score produced by a classifier. This manipulation is realized in an anti-adversarial manner, which perturbs the original images along pixel gradients in the opposite direction from… 

Figures and Tables from this paper

Anti-Adversarially Manipulated Attributions for Weakly Supervised Semantic Segmentation and Object Localization
This work enhances non-discriminative yet class-relevant features, which used to make an insufficient contribution to previous attribution maps, so that the resulting AdvCAM identifies more regions of the target object.
Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation
This work proposes a method to reduce the information bottleneck by removing the last activation function of a deep neural network, and introduces a new pooling method that further encourages the transmission of information from non-discriminative regions to the classification.
Towards Noiseless Object Contours for Weakly Supervised Semantic Segmentation
A SANCE model is trained which utilizes an auxiliary segmentation module to sup-plement high-level semantic information for contour training by backbone feature sharing and online label supervision and provides more accurate localization map than CAM for pseudo label generation.
Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast
Weakly-supervised pixel-to-prototype contrast that can provide pixel-level supervisory signals to narrow the gap between classification and segmentation is proposed and seamlessly incorporated into existing WSSS models without any changes to the base networks and does not incur any extra inference burden.
Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Pooling
The second drawback of CAMs is alleviated with the proposed hybrid pooling, which incorporates the global average and local discriminative information to simultaneously ensure object completeness and reduce background regions.
RecurSeed and CertainMix for Weakly Supervised Semantic Segmentation
RecurSeed is proposed which alternately reduces nonand false-detection through recursive iterations, thereby implicitly finding an optimal junction that minimizes both errors and obtaining a new DA method effectively reflecting object existence reliability through the spatial information.
Re-Attention Transformer for Weakly Supervised Object Localization
A re-attention mechanism termed token refinement transformer (TRT) that captures the object-level semantics to guide the localization well and incorporates the class activation map as the semantically aware input to restrain the attention map to the target object.
Activation Modulation and Recalibration Scheme for Weakly Supervised Semantic Segmentation
A novel activation modulation and recalibration scheme that establishes a new state-of-the-art performance on the PASCAL VOC 2012 dataset, surpassing not only current methods trained with the image-level of supervision but also some methods relying on stronger supervision, such as saliency label.
Interactive Semantic Segmentation With Weak Supervision
An novelty interactive image segmentation method based on weak supervision is proposed, which allows multiple feedbacks of easily obtained weakly supervised information and improves the efficiency of utility of the supervision information.


Adversarial Complementary Learning for Weakly Supervised Object Localization
This work mathematically proves that class localization maps can be obtained by directly selecting the class-specific feature maps of the last convolutional layer, which paves a simple way to identify object regions and presents a simple network architecture including two parallel-classifiers for object localization.
Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach
This work investigates a principle way to progressively mine discriminative object regions using classification networks to address the weakly-supervised semantic segmentation problems and proposes a new adversarial erasing approach for localizing and expanding object regions progressively.
On the Robustness of Semantic Segmentation Models to Adversarial Attacks
This paper presents what to their knowledge is the first rigorous evaluation of adversarial attacks on modern semantic segmentation models, using two large-scale datasets and shows how mean-field inference in deep structured models and multiscale processing naturally implement recently proposed adversarial defenses.
Weakly-Supervised Semantic Segmentation via Sub-Category Exploration
This work proposes a simple yet effective approach that introduces a self-supervised task by exploiting the sub- category information and performs clustering on image features to generate pseudo sub-categories labels within each annotated parent class.
FickleNet: Weakly and Semi-Supervised Semantic Image Segmentation Using Stochastic Inference
FickleNet explores diverse combinations of locations on feature maps created by generic deep neural networks and implicitly learns the coherence of each location in the feature maps, resulting in a localization map which identifies both discriminative and other parts of objects.
Learning Pixel-Level Semantic Affinity with Image-Level Supervision for Weakly Supervised Semantic Segmentation
  • Jiwoon Ahn, Suha Kwak
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
On the PASCAL VOC 2012 dataset, a DNN learned with segmentation labels generated by the method outperforms previous models trained with the same level of supervision, and is even as competitive as those relying on stronger supervision.
Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations
IRNet is proposed, which estimates rough areas of individual instances and detects boundaries between different object classes and enables to assign instance labels to the seeds and to propagate them within the boundaries so that the entire areas of instances can be estimated accurately.
Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation
It is found that varying dilation rates can effectively enlarge the receptive fields of convolutional kernels and more importantly transfer the surrounding discriminative information to non-discriminative object regions, promoting the emergence of these regions in the object localization maps.
Mixup-CAM: Weakly-supervised Semantic Segmentation via Uncertainty Regularization
A principled and end-to-end train-able framework to allow the network to pay attention to other parts of the object, while producing a more complete and uniform response map.
Causal Intervention for Weakly-Supervised Semantic Segmentation
A structural causal model to analyze the causalities among images, contexts, and class labels is proposed and a new method: Context Adjustment (CONTA) is developed, to remove the confounding bias in image-level classification and thus provide better pseudo-masks as ground-truth for the subsequent segmentation model.