Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation

@article{Tavera2021PixelbyPixelCA,
  title={Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation},
  author={A. Tavera and Fabio Cermelli and Carlo Masone and Barbara Caputo},
  journal={2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year={2021},
  pages={1959-1968}
}
In this paper we consider the task of semantic segmentation in autonomous driving applications. Specifically, we consider the cross-domain few-shot setting where training can use only few real-world annotated images and many annotated synthetic images. In this context, aligning the domains is made more challenging by the pixel-wise class imbalance that is intrinsic in the segmentation and that leads to ignoring the underrepresented classes and overfitting the well represented ones. We address… 
[PDF] Semantic Reader
5 Citations
Background Citations
3

Figures and Tables from this paper

5 Citations

Hierarchical Instance Mixing across Domains in Aerial Segmentation

A new mixing strategy for aerial segmentation across domains called Hierarchical Instance Mixing (HIMix), which extracts a set of connected components from each semantic mask and mixes them according to a semantic hier- archy and a twin-head architecture in which two separate segmentation heads are fed with variations of the same images in a contrastive fashion to produce segmentation maps.

Convolutional Transformer-Based Few-Shot Learning for Cross-Domain Hyperspectral Image Classification

Experiments indicate that the proposed novel convolutional transformer-based few-shot learning (CTFSL) method is superior to the state-of-the-art cross-domain FSL methods and several typical HSI classification methods in terms of classification accuracy.

Augmentation Invariance and Adaptive Sampling in Semantic Segmentation of Agricultural Aerial Images

A set of suitable augmentation and a consistency loss are used to guide the model to learn semantic representations that are invariant to the photometric and geometric shifts typical of the top-down perspective (Augmentation Invariance).

Improving Generalization in Federated Learning by Seeking Flat Minima

This work investigates behavior through the lens of geometry of the loss and Hessian eigenspectrum, linking the model’s lack of generalization capacity to the sharpness of the solution and shows that training clients locally with Sharpness-Aware Minimization or its adaptive version can sub-stantially improve generalization in Federated Learning and help bridging the gap with centralized models.

MGNet: Mutual-guidance network for few-shot semantic segmentation

References

SHOWING 1-10 OF 57 REFERENCES

ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation

This work proposes two novel, complementary methods using (i) entropy loss and (ii) adversarial loss respectively for unsupervised domain adaptation in semantic segmentation with losses based on the entropy of the pixel-wise predictions.

FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation

This paper introduces the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems, and outperforms baselines across different settings on multiple large-scale datasets.

CANet: Class-Agnostic Segmentation Networks With Iterative Refinement and Attentive Few-Shot Learning

Canet is presented, a class-agnostic segmentation network that performs few-shot segmentation on new classes with only a few annotated images available, and introduces an attention mechanism to effectively fuse information from multiple support examples under the setting of k-shot learning.

DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation

Dual Channel-wise Alignment Networks (DCAN) are presented, a simple yet effective approach to reduce domain shift at both pixel-level and feature-level in deep neural networks for semantic segmentation.

FDA: Fourier Domain Adaptation for Semantic Segmentation

A simple method for unsupervised domain adaptation, whereby the discrepancy between the source and target distributions is reduced by swapping the low-frequency spectrum of one with the other, which results indicate that even simple procedures can discount nuisance variability in the data that more sophisticated methods struggle to learn away.

Few-Shot Adversarial Domain Adaptation

This work provides a framework for addressing the problem of supervised domain adaptation with deep models by carefully designing a training scheme whereby the typical binary adversarial discriminator is augmented to distinguish between four different classes.

Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation

A category-level adversarial network is introduced, aiming to enforce local semantic consistency during the trend of global alignment, to take a close look at the category- level data distribution and align each class with an adaptive adversarial loss.

The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes

This paper generates a synthetic collection of diverse urban images, named SYNTHIA, with automatically generated class annotations, and conducts experiments with DCNNs that show how the inclusion of SYnTHIA in the training stage significantly improves performance on the semantic segmentation task.

IDDA: A Large-Scale Multi-Domain Dataset for Autonomous Driving

A new large scale, synthetic dataset for semantic segmentation with more than 100 different source visual domains is created to explicitly address the challenges of domain shift between training and test data in various weather and view point conditions.

Few-Shot Semantic Segmentation with Prototype Learning

A generalized framework for few-shot semantic segmentation with an alternative training scheme based on prototype learning and metric learning is proposed, which outperforms the baselines by a large margin and shows comparable performance for 1-way few- shot semantic segmentsation on PASCAL VOC 2012 dataset.
...