Learning to Adapt Structured Output Space for Semantic Segmentation

@article{Tsai2018LearningTA,
  title={Learning to Adapt Structured Output Space for Semantic Segmentation},
  author={Yi-Hsuan Tsai and Wei-Chih Hung and Samuel Schulter and Kihyuk Sohn and Ming-Hsuan Yang and Manmohan Chandraker},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={7472-7481}
}
Convolutional neural network-based approaches for semantic segmentation rely on supervision with pixel-level ground truth, but may not generalize well to unseen image domains. [] Key Method Considering semantic segmentations as structured outputs that contain spatial similarities between the source and target domains, we adopt adversarial learning in the output space.
View on IEEE
View With Semantic Reader
719 Citations
Highly Influential Citations
205
Background Citations
391
Methods Citations
359
Results Citations
35

Figures and Tables from this paper

719 Citations

Citation Type

Citation Type

Category-Level Adversaries for Semantic Domain Adaptation
TLDR
This work proposes a generative adversarial network (GAN)-based framework called category-level adversarial adaptation networks (CAA-Nets) for domain adaptation in the context of semantic segmentation and constructs an image-based generator and discriminator pair that can achieve competitive performance compared with some existing methods.
SSF-DAN: Separated Semantic Feature Based Domain Adaptation Network for Semantic Segmentation
TLDR
A Separated Semantic Feature based domain adaptation network, named SSF-DAN, for semantic segmentation, designed to independently adapt semantic features across the target and source domains and demonstrates robust performance, superior to state-of-the-art methods on benchmark datasets.
Domain Adaptive Semantic Segmentation Through Structure Enhancement
TLDR
This work proposes an effective method to adapt the segmentation network trained on synthetic images to real scenarios in an unsupervised fashion and enhances the structure information of the target images at both the feature level and the output level to improve the adaptation performance for semantic segmentation.
DISCRIMINATIVE PATCH REPRESENTATIONS
TLDR
A domain adaptation method to adapt the source data to the unlabeled target domain and use an adversarial learning scheme to push the feature representations in target patches to the closer distributions in source ones to achieve state-of-the-art performance on semantic segmentation.
Semantic-aware short path adversarial training for cross-domain semantic segmentation
Domain Adaptation for Structured Output via Discriminative Patch Representations
TLDR
A domain adaptation method to adapt the source data to the unlabeled target domain by discovering multiple modes of patch-wise output distribution through the construction of a clustered space and using an adversarial learning scheme to push the feature representations of target patches in the clustered space closer to the distributions of source patches.
Adapting Semantic Segmentation of Urban Scenes via Mask-Aware Gated Discriminator
TLDR
This work proposes a method for domain adaptation that models and adapts foreground objects and background objects separately and demonstrates in experiments that the model outperforms several state-of-the-art baselines in terms of mean intersection over union (mIoU).
Joint Adversarial Learning for Domain Adaptation in Semantic Segmentation
TLDR
This work proposes a novel joint adversarial learning (JAL) to boost the domain discriminator in output space by introducing the information ofdomain discriminator from low-level features and proposes a weight transfer module (WTM) to alleviate the inherent bias of the trained decoder towards source domain.
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
TLDR
This work proposes two novel, complementary methods using (i) entropy loss and (ii) adversarial loss respectively for unsupervised domain adaptation in semantic segmentation with losses based on the entropy of the pixel-wise predictions.
Large Scale Unsupervised Domain Adaptation of Segmentation Networks with Adversarial Learning
TLDR
The core concept is to infuse fully convolutional neural networks and adversarial networks for semantic segmentation assuming the structures in the scene and objects of interest are similar in two set of images.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 44 REFERENCES
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
TLDR
This paper introduces the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems, and outperforms baselines across different settings on multiple large-scale datasets.
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
TLDR
This generative adversarial network (GAN)-based method adapts source-domain images to appear as if drawn from the target domain, and outperforms the state-of-the-art on a number of unsupervised domain adaptation scenarios by large margins.
No More Discrimination: Cross City Adaptation of Road Scene Segmenters
TLDR
This work proposes an unsupervised learning approach to adapt road scene segmenters across different cities by advancing a joint global and class-specific domain adversarial learning framework, and shows that this method improves the performance of semantic segmentation in multiple cities across continents.
Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
TLDR
This work proposes a curriculum-style learning approach to minimize the domain gap in semantic segmentation, which significantly outperforms the baselines as well as the only known existing approach to the same problem.
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
  • Jifeng Dai, Kaiming He, Jian Sun
  • Computer Science
    2015 IEEE International Conference on Computer Vision (ICCV)
  • 2015
TLDR
This paper proposes a method that achieves competitive accuracy but only requires easily obtained bounding box annotations, and yields state-of-the-art results on PASCAL VOC 2012 and PASCal-CONTEXT.
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
TLDR
This work shows how to improve semantic segmentation through the use of contextual information, specifically, ' patch-patch' context between image regions, and 'patch-background' context, and formulate Conditional Random Fields with CNN-based pairwise potential functions to capture semantic correlations between neighboring patches.
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
TLDR
A novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model that adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs is proposed.
Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
TLDR
Expectation-Maximization (EM) methods for semantic image segmentation model training under weakly supervised and semi-supervised settings are developed and extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentsation benchmark, while requiring significantly less annotation effort.
The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes
TLDR
This paper generates a synthetic collection of diverse urban images, named SYNTHIA, with automatically generated class annotations, and conducts experiments with DCNNs that show how the inclusion of SYnTHIA in the training stage significantly improves performance on the semantic segmentation task.
Fully Convolutional Networks for Semantic Segmentation
TLDR
It is shown that convolutional networks by themselves, trained end- to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation.
...
1
2
3
4
5
...