Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation

@article{Wang2020AlleviatingSS,
  title={Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation},
  author={Zhonghao Wang and Yunchao Wei and Rog{\'e}rio Schmidt Feris and Jinjun Xiong and Wen-mei W. Hwu and Thomas S. Huang and Humphrey Shi},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
  year={2020},
  pages={4043-4047}
}
Utilizing synthetic data for semantic segmentation can significantly relieve human efforts in labelling pixel-level masks. A key challenge of this task is how to alleviate the data distribution discrepancy between the source and target domains, i.e. reducing domain shift. The common approach to this problem is to minimize the discrepancy between feature distributions from different domains through adversarial training. However, directly aligning the feature distribution globally cannot… 

Figures and Tables from this paper

Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learning
TLDR
This work introduces a novel approach to unsupervised and semi-supervised domain adaptation for semantic segmentation that leverages contrastive learning to bridge the domain gap by aligning the features of structurally similar label patches across domains.
Cross-modal Learning for Domain Adaptation in 3D Semantic Segmentation
TLDR
This work proposes cross-modal learning, where it enforce consistency between the predictions of two modalities via mutual mimicking, to enable domain adaptation of semantic segmentation from either the 2D image, the 3D point cloud or from both.
Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bank
TLDR
The key element of this approach is the contrastive learning module that enforces the segmentation network to yield similar pixel-level feature representations for same-class samples across the whole dataset, maintaining a memory bank which is continuously updated with relevant and high-quality feature vectors from labeled data.
Dual-Teacher++: Exploiting Intra-Domain and Inter-Domain Knowledge With Reliable Transfer for Cardiac Segmentation
TLDR
This paper proposes a cutting-edge semi-supervised domain adaptation framework, namely Dual-Teacher++, and designs novel dual teacher models, including an inter-domain teacher model to explore cross-modality priors from source domain and an intra-domain teachers model to investigate the knowledge beneath unlabeled target domain.
CCNet: Criss-Cross Attention for Semantic Segmentation
TLDR
This work proposes a Criss-Cross Network (CCNet) for obtaining contextual information in a more effective and efficient way and achieves the mIoU score of 81.4 and 45.22 on Cityscapes test set and ADE20K validation set, respectively, which are the new state-of-the-art results.
Active learning using weakly supervised signals for quality inspection
TLDR
This work develops a methodology for learning actively from rapidly mined, weakly annotated data, enabling a fast, direct feedback from the operators on the production line and tackling a big machine vision weakness: false positives.
Adaptive Consistency Regularization for Semi-Supervised Transfer Learning
TLDR
This work considers semi-supervised learning and transfer learning jointly, leading to a more practical and competitive paradigm that can utilize both powerful pre-trained models from source domain as well as labeled/unlabeled data in the target domain.
Synthetic training data generation for deep learning based quality inspection
TLDR
This work presents a generic simulation pipeline to render images of defective or healthy (non defective) parts, and designs a texture scanning and generation method, demonstrating that simulations can complement real images to boost performances.
The 1st Agriculture-Vision Challenge: Methods and Results
  • M. Chiu, Xingqian Xu, Jianyu Tang
  • Computer Science
    2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • 2020
The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images, especially for the semantic
DistillAdapt: Source-Free Active Visual Domain Adaptation
TLDR
The source-free approach, DistillAdapt, results in an improvement of 0 .
...
...

References

SHOWING 1-10 OF 41 REFERENCES
Learning to Adapt Structured Output Space for Semantic Segmentation
TLDR
A multi-level adversarial network is constructed to effectively perform output space domain adaptation at different feature levels and it is shown that the proposed method performs favorably against the state-of-the-art methods in terms of accuracy and visual quality.
Unsupervised Domain Adaptation for Semantic Segmentation with GANs
TLDR
This work proposes an approach based on Generative Adversarial Networks (GANs) that brings the embeddings closer in the learned feature space and can achieve state of the art results on two challenging scenarios of synthetic to real domain adaptation.
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
TLDR
This paper introduces the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems, and outperforms baselines across different settings on multiple large-scale datasets.
Weakly Supervised Scene Parsing with Point-based Distance Metric Learning
TLDR
This paper proposes a Point-based Distance Metric Learning (PDML), which leverages semantic relationship among the annotated points by encouraging the feature representations of the intra- and inter-category points to keep consistent.
Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach
TLDR
This work investigates a principle way to progressively mine discriminative object regions using classification networks to address the weakly-supervised semantic segmentation problems and proposes a new adversarial erasing approach for localizing and expanding object regions progressively.
Fully Convolutional Adaptation Networks for Semantic Segmentation
TLDR
FCAN is presented, a novel deep architecture for semantic segmentation which combines Appearance Adaptation Networks (AAN) and Representation Adaptation networks (RAN), which learns a transformation from one domain to the other in the pixel space and RAN is optimized in an adversarial learning manner to maximally fool the domain discriminator with the learnt source and target representations.
Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation
TLDR
The reason why the current adversarial loss is often unstable in minimizing the distribution discrepancy is revealed and it is shown that the method can help ease this issue by minimizing the most similar stuff and instance features between the source and the target domains.
DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
TLDR
Dual Channel-wise Alignment Networks (DCAN) are presented, a simple yet effective approach to reduce domain shift at both pixel-level and feature-level in deep neural networks for semantic segmentation.
ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes
TLDR
This work proposes a new reality oriented adaptation approach for urban scene semantic segmentation by learning from synthetic data that takes advantage of the intrinsic spatial structure presented in urban scene images, and proposes a spatial-aware adaptation scheme to effectively align the distribution of two domains.
Bidirectional Learning for Domain Adaptation of Semantic Segmentation
TLDR
A self-supervised learning algorithm to learn a better segmentation adaptation model and in return improve the image translation model and the bidirectional learning framework for domain adaptation of segmentation is proposed.
...
...