Corpus ID: 220380959

Rate-Regularization and Generalization in VAEs

  title={Rate-Regularization and Generalization in VAEs},
  author={Alican Bozkurt and B. Esmaeili and D. Brooks and Jennifer G. Dy and Jan-Willem van de Meent},
  journal={arXiv: Learning},
  • Alican Bozkurt, B. Esmaeili, +2 authors Jan-Willem van de Meent
  • Published 2019
  • Computer Science, Mathematics
  • arXiv: Learning
  • Variational autoencoders (VAEs) optimize an objective that comprises a reconstruction loss (the distortion) and a KL term (the rate). The rate is an upper bound on the mutual information, which is often interpreted as a regularizer that controls the degree of compression. We here examine whether inclusion of the rate term also improves generalization. We perform rate-distortion analyses in which we control the strength of the rate term, the network capacity, and the difficulty of the… CONTINUE READING


    On Implicit Regularization in β-VAEs
    • 6
    • PDF
    On Implicit Regularization in $\beta$-VAEs
    • 3
    • PDF
    Amortized Inference Regularization
    • 27
    • Highly Influential
    • PDF
    Fixing a Broken ELBO
    • 225
    • Highly Influential
    • PDF
    Variational Autoencoders for Collaborative Filtering
    • 251
    • PDF
    beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
    • 1,444