• Corpus ID: 220347682

In Search of Lost Domain Generalization

@article{Gulrajani2020InSO,
  title={In Search of Lost Domain Generalization},
  author={Ishaan Gulrajani and David Lopez-Paz},
  journal={ArXiv},
  year={2020},
  volume={abs/2007.01434}
}
The goal of domain generalization algorithms is to predict well on distributions different from those seen during training. While a myriad of domain generalization algorithms exist, inconsistencies in experimental conditions -- datasets, architectures, and model selection criteria -- render fair and realistic comparisons difficult. In this paper, we are interested in understanding how useful domain generalization algorithms are in realistic settings. As a first step, we realize that model… 

Tables from this paper

Failure Modes of Domain Generalization Algorithms

It is shown that the largest contributor to the generalization error varies across methods, datasets, regularization strengths and even training lengths, and it is hypothesized that focusing instead on tuning the classifier on top of a rich representation can be a promising direction.

Domain Generalization Needs Stochastic Weight Averaging for Robustness on Domain Shifts

To estimate the center, stochastic weight averaging (SWA) is employed and theoretical analysis describing how SWA supports the generalization bound for an unseen domain and achieves state-of-theart performances over all of widely used domain generalization benchmarks.

Model-Based Domain Generalization

This paper proposes a novel approach for the domain generalization problem called Model-Based Domain Generalization, which uses unlabeled data from the training domains to learn multi-modal domain transformation models that map data from one training domain to any other domain.

Adaptive Methods for Real-World Domain Generalization

This work proposes a domain-adaptive approach consisting of two steps: a) the authors first learn a discriminative domain embedding from unsupervised training examples, and b) use thisdomain embedding as supplementary information to build adomain- Adaptive model, that takes both the input as well as its domain into account while making predictions.

Improving Multi-Domain Generalization through Domain Re-labeling

A general approach for multi-domain generalization, MulDEns, is introduced that uses an ERM-based deep ensembling backbone and performs implicit domain re-labeling through a meta-optimization algorithm, and consistently outperforms ERM by significant margins.

Towards Data-Free Domain Generalization

DKAN is proposed, an approach that extracts and fuses domain-specific knowledge from the available teacher models into a student model robust to domain shift, and achieves the first state-of-the-art results in DFDG by outperforming data-free knowledge distillation and ensemble baselines.

Adaptive Methods for Aggregated Domain Generalization

A domain-adaptive approach to domain generalization that operates in two steps: the authors cluster training data within a carefully chosen feature space to create pseudodomains, and using these pseudo-domains they learn a domain- Adaptive classifier that makes predictions using information about both the input and the pseudo-domain it belongs to.

Automated Domain Discovery from Multiple Sources to Improve Zero-Shot Generalization

This paper proposes (i) Group-DRO++, which incorporates an explicit clustering step to identify custom domains in an existing DRO technique; and (ii) DReaME, which produces effective multi-domain ensembles through implicit domain re-labeling with a novel meta-optimization algorithm.

Discovery of New Multi-Level Features for Domain Generalization via Knowledge Corruption

This work proposes COLUMBUS, a method that enforces new feature discovery via a targeted corruption of the most relevant input and multilevel representations of the data that achieves new state-of-the-art results by outperforming 18 DG algorithms on multiple DG benchmark datasets in the DomainBed framework.

Evolving Domain Generalization

The theoretical result reveals the benefits of modeling the relation between two consecutive tasks by learning a globally consistent directional mapping function and suggests solving the EDG problem in a meta-learning manner, which leads to directional prototypical network, the first method for theEDG problem.
...

References

SHOWING 1-10 OF 134 REFERENCES

Learning to Balance Specificity and Invariance for In and Out of Domain Generalization

This work introduces Domain-specific Masks for Generalization, a model for improving both in-domain and out-of-domain generalization performance and encourages the masks to learn a balance of domain-invariant and domain-specific features, thus enabling a model which can benefit from the predictive power of specialized features while retaining the universal applicability of Domain-Invariant features.

Feature-Critic Networks for Heterogeneous Domain Generalization

This work considers a more challenging setting of heterogeneous domain generalisation, where the unseen domains do not share label space with the seen ones, and the goal is to train a feature representation that is useful off theshelf for novel data and novel categories.

Unshuffling Data for Improved Generalization

This work describes a training procedure to capture the patterns that are stable across environments while discarding spurious ones, and demonstrates multiple use cases with the task of visual question answering, which is notorious for dataset biases.

Generalizing Across Domains via Cross-Gradient Training

Empirical evaluation on three different applications establishes that (1) domain-guided perturbation provides consistently better generalization to unseen domains, compared to generic instance perturbations methods, and that (2) data augmentation is a more stable and accurate method than domain adversarial training.

Domain Generalization by Marginal Transfer Learning

This work lays the learning theoretic foundations of domain generalization, building on the earlier conference paper where the problem of DG was introduced, and presents two formal models of data generation, corresponding notions of risk, and distribution-free generalization error analysis.

Best Sources Forward: Domain Generalization through Source-Specific Nets

This work designs a deep network with multiple domain-specific classifiers, each associated to a source domain, and introduced a domain agnostic component supporting the final classifier.

A Survey of Unsupervised Deep Domain Adaptation

A survey will compare single-source and typically homogeneous unsupervised deep domain adaptation approaches, combining the powerful, hierarchical representations from deep learning with domain adaptation to reduce reliance on potentially costly target data labels.

Deeper, Broader and Artier Domain Generalization

This paper builds upon the favorable domain shift-robust properties of deep learning methods, and develops a low-rank parameterized CNN model for end-to-end DG learning that outperforms existing DG alternatives.

Sequential Learning for Domain Generalization

This paper proposes a sequential learning framework for Domain Generalization (DG), the problem of training a model that is robust to domain shift by design, and shows that for MLDG it leads to a simple to implement and fast algorithm that provides consistent performance improvement on a variety of DG benchmarks.

Episodic Training for Domain Generalization

Using the Visual Decathlon benchmark, it is demonstrated that the episodic-DG training improves the performance of such a general purpose feature extractor by explicitly training a feature for robustness to novel problems, showing that DG training can benefit standard practice in computer vision.
...