• Corpus ID: 54089884

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

@article{Locatello2019ChallengingCA,
  title={Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations},
  author={Francesco Locatello and Stefan Bauer and Mario Lucic and Sylvain Gelly and Bernhard Sch{\"o}lkopf and Olivier Bachem},
  journal={ArXiv},
  year={2019},
  volume={abs/1811.12359}
}
The key idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms. [] Key Method We first theoretically show that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data.
View PDF on arXiv
View With Semantic Reader
696 Citations
Highly Influential Citations
125
Background Citations
470
Methods Citations
163
Results Citations
43

696 Citations

Citation Type

Citation Type

Disentangled Representations from Non-Disentangled Models
TLDR
This paper proposes to extract disentangled representations from the state-ofthe-art generative models trained without disentangling terms in their objectives, and employs little or no hyperparameters when learning representations while achieving results on par with existing state of theart models.
Disentangling Factors of Variation Using Few Labels
TLDR
Overall, this paper empirically validate that with little and imprecise supervision it is possible to reliably learn disentangled representations.
A Heuristic for Unsupervised Model Selection for Variational Disentangled Representation Learning
TLDR
The approach, Unsupervised Disentanglement Ranking (UDR), leverages the recent theoretical results that explain why variational autoencoders disentangle, to quantify the quality of disentangled representations by performing pairwise comparisons between trained model representations.
Group-disentangled Representation Learning with Weakly-Supervised Regularization
TLDR
GroupVAE is proposed, a simple yet effective Kullback-Leibler divergence-based regularization across shared latent representations to enforce consistent and disentangled representations that improve upon downstream tasks, including fair classification and 3D shape-related tasks.
Unsupervised learning of disentangled representations in deep restricted kernel machines with orthogonality constraints
Theory and Evaluation Metrics for Learning Disentangled Representations
TLDR
This work characterize the concept "disentangled representations" used in supervised and unsupervised methods along three dimensions-informativeness, separability and interpretability - which can be expressed and quantified explicitly using information-theoretic constructs.
Is Independence all you need? On the Generalization of Representations Learned from Correlated Data
TLDR
This work bridges the gap to real-world scenarios by analyzing the behavior of most prominent methods and disentanglement scores on correlated data in a large scale empirical study (including 3900 models).
WeLa-VAE: Learning Alternative Disentangled Representations Using Weak Labels
TLDR
This paper considers weak supervision by means of high-level labels that are not assumed to be explicitly related to the ground truth factors, which involves the maximization of a modified variational lower bound and total correlation regularization in WeLa-VAE, a variational inference framework where observations and labels share the same latent variables.
Robust Disentanglement of a Few Factors at a Time
TLDR
This work introduces population-based training (PBT) for improving consistency in training variational autoencoders (VAEs) and demonstrates the validity of this approach in a supervised setting and introduces the recursive rPU-VAE approach, which shows striking improvement in state-of-the-art unsupervised disentanglement performance and robustness across multiple datasets and metrics.
Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations
TLDR
It is demonstrated that the proposed prior significantly mitigates the trade-off between reconstruction loss and disentanglement over the state of the art and resolves the problem of unidentifiability of the standard VAE normal prior.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 68 REFERENCES
Learning Deep Disentangled Embeddings with the F-Statistic Loss
TLDR
A new paradigm for discovering disentangled representations of class structure is proposed and a novel loss function based on the $F$ statistic is proposed, which describes the separation of two or more distributions.
Learning Disentangled Representations with Semi-Supervised Deep Generative Models
TLDR
This work proposes to learn disentangled representations that encode distinct aspects of the data into separate variables using model architectures that generalise from standard VAEs, employing a general graphical model structure in the encoder and decoder.
A Framework for the Quantitative Evaluation of Disentangled Representations
TLDR
A framework for the quantitative evaluation of disentangled representations when the ground-truth latent structure is available is proposed and three criteria are explicitly defined and quantified to elucidate the quality of learnt representations and thus compare models on an equal basis.
Interventional Robustness of Deep Latent Variable Models
TLDR
The interventional robustness score is introduced, which provides a quantitative evaluation of robustness in learned representations with respect to interventions on generative factors and changing nuisance factors, and how this score can be estimated from labeled observational data, that may be confounded, and further provide an efficient algorithm that scales linearly in the dataset size.
beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
Learning an interpretable factorised representation of the independent data generative factors of the world without supervision is an important precursor for the development of artificial…
Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
TLDR
The Multi-Level Variational Autoencoder (ML-VAE), a new deep probabilistic model for learning a disentangled representation of a set of grouped observations, separates the latent representation into semantically meaningful parts by working both at the group level and the observation level, while retaining efficient test-time inference.
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations
TLDR
This work considers the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and proposes a variational inference based approach to inferdisentangled latent factors.
Competitive Training of Mixtures of Independent Deep Generative Models
TLDR
This work considers mixtures of implicit generative models that ``disentangle'' the independent generative mechanisms underlying the data and proposes a competitive training procedure in which the models only need to capture the portion of the data distribution from which they can produce realistic samples.
Recent Advances in Autoencoder-Based Representation Learning
TLDR
An in-depth review of recent advances in representation learning with a focus on autoencoder-based models and makes use of meta-priors believed useful for downstream tasks, such as disentanglement and hierarchical organization of features.
Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness
TLDR
A causal perspective on representation learning is provided which covers disentanglement and domain shift robustness as special cases and a new metric for the quantitative evaluation of deep latent variable models is introduced.
...
1
2
3
4
5
...