DOT-VAE: Disentangling One Factor at a Time

  title={DOT-VAE: Disentangling One Factor at a Time},
  author={Vaishnavi Patil and Matthew Evanusa and Joseph J{\'a}J{\'a}},
. As we enter the era of machine learning characterized by an overabundance of data, discovery, organization, and interpretation of the data in an unsupervised manner becomes a critical need. One promising approach to this endeavour is the problem of Disentanglement , which aims at learning the underlying generative latent factors, called the factors of variation, of the data and encoding them in disjoint latent representations. Recent advances have made efforts to solve this problem for… 



beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

Learning an interpretable factorised representation of the independent data generative factors of the world without supervision is an important precursor for the development of artificial

InfoGAN-CR and ModelCentrality: Self-supervised Model Training and Selection for Disentangling GANs

This work designs a novel approach for training disentangled GANs with self-supervision and proposes contrastive regularizer, which is inspired by a natural notion of disentanglement: latent traversal, and proposes an unsupervised model selection scheme called ModelCentrality, which uses generated synthetic samples to compute the medoid of a collection of models.

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

This paper theoretically shows that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data, and trains more than 12000 models covering most prominent methods and evaluation metrics on seven different data sets.

Learning Disentangled Representations with Latent Variation Predictability

The proposed variation predictability is a general constraint that is applicable to the VAE and GAN frameworks for boosting disentangled latent representations and correlates well with existing ground-truth-required metrics and the proposed algorithm is effective for disentanglement learning.

Learning Deep Disentangled Embeddings with the F-Statistic Loss

A new paradigm for discovering disentangled representations of class structure is proposed and a novel loss function based on the $F$ statistic is proposed, which describes the separation of two or more distributions.

Learning Discrete and Continuous Factors of Data via Alternating Disentanglement

A simple procedure for minimizing the total correlation of the continuous latent variables without having to use a discriminator network or perform importance sampling, via cascading the information flow in the $\beta$-vae framework is shown.

Understanding disentangling in β-VAE

A modification to the training regime of β-VAE is proposed, that progressively increases the information capacity of the latent code during training, to facilitate the robust learning of disentangled representations in β- VAE, without the previous trade-off in reconstruction accuracy.

Isolating Sources of Disentanglement in Variational Autoencoders

We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables. We use this to motivate our $\beta$-TCVAE (Total Correlation

OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization

It is shown that GANs have a natural advantage in disentangling with an alternating latent variable (noise) sampling method that is straightforward and robust and a brand-new perspective on designing the structure of the generator and discriminator is provided, demonstrating that a minor structural change and an orthogonal regularization on model weights entails an improved disentanglement.

Disentangling by Factorising

FactorVAE, a method that disentangles by encouraging the distribution of representations to be factorial and hence independent across the dimensions, is proposed and it improves upon $\beta$-VAE by providing a better trade-off between disentanglement and reconstruction quality.