# Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness

@article{Shen2021RegularizingVA,
title={Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness},
author={Dazhong Shen and Chuan Qin and Chao Wang and Hengshu Zhu and Enhong Chen and Hui Xiong},
journal={ArXiv},
year={2021},
volume={abs/2110.12381}
}
• Published 1 August 2021
• Computer Science
• ArXiv
As one of the most popular generative models, Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. However, when the decoder network is sufficiently expressive, VAE may lead to posterior collapse; that is, uninformative latent representations may be learned. To this end, in this paper, we propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space, and thus the representation can be learned…
1 Citations

## Figures and Tables from this paper

• Computer Science
NeurIPS
• 2021
This paper revisits the task of topic modeling by transforming each document into a directed graph with word dependency as edges between word nodes, and develops a novel approach, namely Graph Neural Topic Model (GNTM).

## References

SHOWING 1-10 OF 42 REFERENCES

• Computer Science
ACL
• 2020
Batch Normalized-VAe (BN-VAE), a simple but effective approach to set a lower bound of the expectation by regularizing the distribution of the approximate posterior’s parameters, surpasses strong autoregressive baselines on language modeling, text classification and dialogue generation, and rivals more complex approaches while keeping almost the same training time as VAE.
• Computer Science
ICLR
• 2019
This paper investigates posterior collapse from the perspective of training dynamics and proposes an extremely simple modification to VAE training to reduce inference lag: depending on the model's current mutual information between latent variable and observation, the inference network is optimized before performing each model update.
• Computer Science
AAAI
• 2019
It is shown that the proposed Info-VAE model can significantly improve the quality of the variational posterior and can make effective use of the latent features regardless of the flexibility of the decoding distribution.
• Computer Science
ICLR
• 2017
This paper presents a simple but principled method to learn global representations by combining Variational Autoencoder (VAE) with neural autoregressive models such as RNN, MADE and PixelRNN/CNN with greatly improve generative modeling performance of VAEs.
• Computer Science
ICLR
• 2019
This paper proposes an alternative that utilizes the most powerful generative models as decoders, whilst optimising the variational lower bound all while ensuring that the latent variables preserve and encode useful information.
• Computer Science
ICLR
• 2017
Learning an interpretable factorised representation of the independent data generative factors of the world without supervision is an important precursor for the development of artificial
• Hao Fu
• Computer Science
NAACL
• 2019
A cyclical annealing schedule is proposed, which simply repeats the process of increasing \beta multiple times, and allows to learn more meaningful latent codes progressively by leveraging the results of previous learning cycles as warm re-restart.
• Computer Science
ICML
• 2015
This work introduces a simple modification for autoencoder neural networks that yields powerful generative models and proves that this approach is competitive with state-of-the-art tractable distribution estimators.
• Computer Science
EMNLP
• 2019
An LVM to directly match the aggregated posterior to the prior is developed, which can be viewed as a natural extension of VAEs with a regularization of maximizing mutual information, mitigating the “posterior collapse” issue.
• Computer Science
EMNLP
• 2017
A novel hybrid architecture that blends fully feed-forward convolutional and deconvolutional components with a recurrent language model is proposed that helps to avoid the issue of the VAE collapsing to a deterministic model.