Corpus ID: 207880653

Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse

@inproceedings{Lucas2019DontBT,
  title={Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse},
  author={James Lucas and George Tucker and Roger B. Grosse and Mohammad Norouzi},
  booktitle={NeurIPS},
  year={2019}
}
  • James Lucas, George Tucker, +1 author Mohammad Norouzi
  • Published in NeurIPS 2019
  • Computer Science, Mathematics
  • Posterior collapse in Variational Autoencoders (VAEs) arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Probabilistic PCA (pPCA). We explain how posterior collapse may occur in pPCA due to local maxima in the log marginal likelihood. Unexpectedly, we prove that the ELBO objective for… CONTINUE READING

    Create an AI-powered research feed to stay up to date with new papers like this posted to ArXiv

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 44 REFERENCES

    Diagnosing and Enhancing VAE Models

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Loss Landscapes of Regularized Linear Autoencoders

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Deep Learning Face Attributes in the Wild

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Auto-Encoding Variational Bayes

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    A

    • A. Razavi
    • van den Oord, B. Poole, and O. Vinyals. Preventing posterior collapse with delta-VAEs. In International Conference on Learning Representations
    • 2019
    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Ladder Variational Autoencoders

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Generating Sentences from a Continuous Space

    VIEW 5 EXCERPTS
    HIGHLY INFLUENTIAL