• Corpus ID: 221655276

# Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations

@article{Cheng2020RevisitingFA,
title={Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations},
author={Ze Cheng and Juncheng Li and Chenxu Wang and Jixuan Gu and Hao Xu and Xinjian Li and Florian Metze},
journal={ArXiv},
year={2020},
volume={abs/2009.05739}
}
• Published 12 September 2020
• Computer Science
• ArXiv
In the problem of learning disentangled representations, one of the promising methods is to factorize aggregated posterior by penalizing the total correlation of sampled latent variables. However, this well-motivated strategy has a blind spot: there is a disparity between the sampled latent representation and its corresponding mean representation. In this paper, we provide a theoretical explanation that low total correlation of sampled representation cannot guarantee low total correlation of…
2 Citations

## Figures and Tables from this paper

### Perioperative Predictions with Interpretable Latent Representation

• Computer Science
KDD
• 2022
The results show that the latent representation provided by cVAE leads to superior performance in classification, regression and multi-task predictions, and the interpretability of the disentangled representation and its capability to capture intrinsic characteristics of hospitalized patients.

### Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoders

• Computer Science
ArXiv
• 2021
This paper extends the existing definition of mean representations to mean representations and shows that active variables are equally disentangled in both representations, and isolates the passive variables, which show that they are responsible for the discrepancies between mean and sampled representations.

## References

SHOWING 1-10 OF 29 REFERENCES

### Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

• Computer Science
ICML
• 2019
This paper theoretically shows that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data, and trains more than 12000 models covering most prominent methods and evaluation metrics on seven different data sets.

### Isolating Sources of Disentanglement in VAEs

• Computer Science
• 2018
A decomposition of the variational lower bound is shown that can be used to explain the success of the β-VAE in learning disentangled representations, and a new information-theoretic disentanglement metric is proposed, which is classifier-free and generalizable to arbitrarily-distributed and non-scalar latent variables.

### Disentangling by Factorising

• Computer Science
ICML
• 2018
FactorVAE, a method that disentangles by encouraging the distribution of representations to be factorial and hence independent across the dimensions, is proposed and it improves upon $\beta$-VAE by providing a better trade-off between disentanglement and reconstruction quality.

### Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

• Computer Science
ICLR
• 2018
This work considers the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and proposes a variational inference based approach to inferdisentangled latent factors.

### Deep Variational Information Bottleneck

• Computer Science
ICLR
• 2017
It is shown that models trained with the VIB objective outperform those that are trained with other forms of regularization, in terms of generalization performance and robustness to adversarial attack.

### Auto-Encoding Variational Bayes

• Computer Science
ICLR
• 2014
A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.

### Representation Learning: A Review and New Perspectives

• Computer Science
IEEE Transactions on Pattern Analysis and Machine Intelligence
• 2013
Recent work in the area of unsupervised feature learning and deep learning is reviewed, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks.

### 3d shapes dataset

• https://github.com/deepmind/3dshapes-dataset/.
• 2018

• 2016

### Isolating Sources of Disentanglement in Variational Autoencoders

• Computer Science
NeurIPS
• 2018
We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables. We use this to motivate our $\beta$-TCVAE (Total Correlation