Semi-supervised Domain Adaptation via Sample-to-Sample Self-Distillation

  title={Semi-supervised Domain Adaptation via Sample-to-Sample Self-Distillation},
  author={Jeongbeen Yoon and Dahyun Kang and Minsu Cho},
  journal={2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
Semi-supervised domain adaptation (SSDA) is to adapt a learner to a new domain with only a small set of labeled samples when a large labeled dataset is given on a source domain. In this paper, we propose a pair-based SSDA method that adapts a model to the target domain using self-distillation with sample pairs. Each sample pair is composed of a teacher sample from a labeled dataset (i.e., source or labeled target) and its student sample from an unlabeled dataset (i.e., unlabeled target). Our… 

Pick up the PACE: Fast and Simple Domain Adaptation via Ensemble Pseudo-Labeling

A fast and simple DA method consisting of three stages: domain alignment by covariance matching, pseudo-labeling, and ensembling that exceeds previous state-of-the-art methods on most benchmark adaptation tasks without training a neural network.

Combating Label Distribution Shift for Active Domain Adaptation

This work considers the problem of active domain adaptation to unlabeled target data, of which subset is actively selected and labeled given a budget constraint, and devise a method that substantially outperforms existing methods in every adaptation scenario.

A Survey on Deep Transfer Learning and Beyond

This survey first review more than 50 representative approaches of DTL in the last decade and systematically summarize them into four categories and divides each category into subcategories according to models, functions, and operation objects.



Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptation

An SSDA framework that aims to align features via alleviation of the intra-domain discrepancy within the target domain is proposed and the incompatibility of the conventional adversarial perturbation methods with SSDA is demonstrated.

Semi-supervised Domain Adaptation with Subspace Learning for visual recognition

A novel domain adaptation framework, named Semi-supervised Domain Adaptation with Subspace Learning (SDASL), which jointly explores invariant low-dimensional structures across domains to correct data distribution mismatch and leverages available unlabeled target examples to exploit the underlying intrinsic information in the target domain.

Cluster Alignment With a Teacher for Unsupervised Domain Adaptation

Cluster Alignment with a Teacher (CAT) is proposed for unsupervised domain adaptation, which can effectively incorporate the discriminative clustering structures in both domains for better adaptation.

Moment Matching for Multi-Source Domain Adaptation

A new deep learning approach, Moment Matching for Multi-Source Domain Adaptation (M3SDA), which aims to transfer knowledge learned from multiple labeled source domains to an unlabeled target domain by dynamically aligning moments of their feature distributions.

Domain-Adversarial Training of Neural Networks

A new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions, which can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer.

DLOW: Domain Flow for Adaptation and Generalization

A domain flow generation model to bridge two different domains by generating a continuous sequence of intermediate domains flowing from one domain to the other and demonstrating the effectiveness of the model for both cross-domain semantic segmentation and the style generalization tasks on benchmark datasets is presented.

Domain Generalization with MixStyle

A novel approach based on probabilistically mixing instancelevel feature statistics of training samples across source domains, motivated by the observation that visual domain is closely related to image style, which results in novel domains being synthesized implicitly and hence the generalizability of the trained model.

Domain-Specific Batch Normalization for Unsupervised Domain Adaptation

A novel unsupervised domain adaptation framework based on domain-specific batch normalization in deep neural networks that achieves the state-of-the-art accuracy in the standard setting and the multi-source domain adaption scenario.

Learning Semantic Representations for Unsupervised Domain Adaptation

Moving semantic transfer network is presented, which learn semantic representations for unlabeled target samples by aligning labeled source centroid and pseudo-labeled target centroid, resulting in an improved target classification accuracy.

Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation

A novel category anchor-guided (CAG) UDA model for semantic segmentation is proposed, which explicitly enforces category-aware feature alignment to learn shared discriminative features and classifiers simultaneously.