Con2DA: Simplifying Semi-supervised Domain Adaptation by Learning Consistent and Contrastive Feature Representations

@article{PerezCarrasco2022Con2DASS,
  title={Con2DA: Simplifying Semi-supervised Domain Adaptation by Learning Consistent and Contrastive Feature Representations},
  author={Manuel P'erez-Carrasco and Pavlos Protopapas and Guillermo Cabrera-Vives},
  journal={ArXiv},
  year={2022},
  volume={abs/2204.01558}
}
In this work, we present Con 2 DA, a simple framework that extends recent advances in semi-supervised learning to the semi-supervised domain adaptation (SSDA) problem. Our framework generates pairs of associated samples by performing stochastic data transformations to a given input. Associated data pairs are mapped to a feature representation space using a feature extractor. We use different loss functions to enforce consistency between the feature representations of associated data pairs of… 
View PDF on arXiv
2 Citations
Background Citations
1
Methods Citations
1

Figures and Tables from this paper

2 Citations

Pick up the PACE: Fast and Simple Domain Adaptation via Ensemble Pseudo-Labeling

A fast and simple DA method consisting of three stages: domain alignment by covariance matching, pseudo-labeling, and ensembling that exceeds previous state-of-the-art methods on most benchmark adaptation tasks without training a neural network.

Addressing Distribution Shift at Test Time in Pre-trained Language Models

This work presents an approach (MEMO-CL) that improves the performance of PLMs at test-time under distribution shift by taking advantage of the latest unsupervised techniques in data augmentation and adaptation to minimize the entropy of the PLM’s output distribution.

References

SHOWING 1-10 OF 34 REFERENCES

Semi-Supervised Domain Adaptation via Minimax Entropy

A novel Minimax Entropy (MME) approach that adversarially optimizes an adaptive few-shot model for semi-supervised domain adaptation (SSDA) setting, setting a new state of the art for SSDA.

Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptation

An SSDA framework that aims to align features via alleviation of the intra-domain discrepancy within the target domain is proposed and the incompatibility of the conventional adversarial perturbation methods with SSDA is demonstrated.

Unsupervised Data Augmentation for Consistency Training

A new perspective on how to effectively noise unlabeled examples is presented and it is argued that the quality of noising, specifically those produced by advanced data augmentation methods, plays a crucial role in semi-supervised learning.

A Simple Framework for Contrastive Learning of Visual Representations

It is shown that composition of data augmentations plays a critical role in defining effective predictive tasks, and introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning.

Supervised Contrastive Learning

A novel training methodology that consistently outperforms cross entropy on supervised learning tasks across different architectures and data augmentations is proposed, and the batch contrastive loss is modified, which has recently been shown to be very effective at learning powerful representations in the self-supervised setting.

Unsupervised Feature Learning via Non-parametric Instance Discrimination

This work forms this intuition as a non-parametric classification problem at the instance-level, and uses noise-contrastive estimation to tackle the computational challenges imposed by the large number of instance classes.

Analysis of Representations for Domain Adaptation

The theory illustrates the tradeoffs inherent in designing a representation for domain adaptation and gives a new justification for a recently proposed model which explicitly minimizes the difference between the source and target domains, while at the same time maximizing the margin of the training set.

Unsupervised Domain Adaptation by Backpropagation

The method performs very well in a series of image classification experiments, achieving adaptation effect in the presence of big domain shifts and outperforming previous state-of-the-art on Office datasets.

Maximum Classifier Discrepancy for Unsupervised Domain Adaptation

A new approach that attempts to align distributions of source and target by utilizing the task-specific decision boundaries by maximizing the discrepancy between two classifiers' outputs to detect target samples that are far from the support of the source.

Moment Matching for Multi-Source Domain Adaptation

A new deep learning approach, Moment Matching for Multi-Source Domain Adaptation (M3SDA), which aims to transfer knowledge learned from multiple labeled source domains to an unlabeled target domain by dynamically aligning moments of their feature distributions.