• Corpus ID: 221655276

Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations

@article{Cheng2020RevisitingFA,
  title={Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations},
  author={Ze Cheng and Juncheng Li and Chenxu Wang and Jixuan Gu and Hao Xu and Xinjian Li and Florian Metze},
  journal={ArXiv},
  year={2020},
  volume={abs/2009.05739}
}
In the problem of learning disentangled representations, one of the promising methods is to factorize aggregated posterior by penalizing the total correlation of sampled latent variables. However, this well-motivated strategy has a blind spot: there is a disparity between the sampled latent representation and its corresponding mean representation. In this paper, we provide a theoretical explanation that low total correlation of sampled representation cannot guarantee low total correlation of… 

Perioperative Predictions with Interpretable Latent Representation

TLDR
The results show that the latent representation provided by cVAE leads to superior performance in classification, regression and multi-task predictions, and the interpretability of the disentangled representation and its capability to capture intrinsic characteristics of hospitalized patients.

Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoders

TLDR
This paper extends the existing definition of mean representations to mean representations and shows that active variables are equally disentangled in both representations, and isolates the passive variables, which show that they are responsible for the discrepancies between mean and sampled representations.

References

SHOWING 1-10 OF 29 REFERENCES

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

TLDR
This paper theoretically shows that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data, and trains more than 12000 models covering most prominent methods and evaluation metrics on seven different data sets.

Isolating Sources of Disentanglement in VAEs

TLDR
A decomposition of the variational lower bound is shown that can be used to explain the success of the β-VAE in learning disentangled representations, and a new information-theoretic disentanglement metric is proposed, which is classifier-free and generalizable to arbitrarily-distributed and non-scalar latent variables.

Disentangling by Factorising

TLDR
FactorVAE, a method that disentangles by encouraging the distribution of representations to be factorial and hence independent across the dimensions, is proposed and it improves upon $\beta$-VAE by providing a better trade-off between disentanglement and reconstruction quality.

Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

TLDR
This work considers the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and proposes a variational inference based approach to inferdisentangled latent factors.

Deep Variational Information Bottleneck

TLDR
It is shown that models trained with the VIB objective outperform those that are trained with other forms of regularization, in terms of generalization performance and robustness to adversarial attack.

Auto-Encoding Variational Bayes

TLDR
A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.

Representation Learning: A Review and New Perspectives

TLDR
Recent work in the area of unsupervised feature learning and deep learning is reviewed, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks.

3d shapes dataset

  • https://github.com/deepmind/3dshapes-dataset/.
  • 2018

Elbo surgery: yet another way to carve up the variational evidence

  • 2016

Isolating Sources of Disentanglement in Variational Autoencoders

We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables. We use this to motivate our $\beta$-TCVAE (Total Correlation