• Corpus ID: 239050174

Controllable and Compositional Generation with Latent-Space Energy-Based Models

@article{Nie2021ControllableAC,
  title={Controllable and Compositional Generation with Latent-Space Energy-Based Models},
  author={Weili Nie and Arash Vahdat and Anima Anandkumar},
  journal={ArXiv},
  year={2021},
  volume={abs/2110.10873}
}
Controllable generation is one of the key requirements for successful adoption of deep generative models in real-world applications, but it still remains as a great challenge. In particular, the compositional ability to generate novel concept combinations is out of reach for most current models. In this work, we use energybased models (EBMs) to handle compositional generation over a set of attributes. To make them scalable to high-resolution image generation, we introduce an EBM in the latent… 
Score-based Generative Modeling in Latent Space
TLDR
The Latent Score-based Generative Model (LSGM) is proposed, a novel approach that trains SGMs in a latent space, relying on the variational autoencoder framework, and achieves state-of-the-art likelihood on the binarized OMNIGLOT dataset.
Improved Input Reprogramming for GAN Conditioning
TLDR
The proposed algorithm INREP+ addresses the existing issues with the novel uses of invertible neural networks and Positive-Unlabeled (PU) learning and outperforms all existing methods, particularly when label information is scarce, noisy, and/or imbalanced.

References

SHOWING 1-10 OF 77 REFERENCES
Score-based Generative Modeling in Latent Space
TLDR
The Latent Score-based Generative Model (LSGM) is proposed, a novel approach that trains SGMs in a latent space, relying on the variational autoencoder framework, and achieves state-of-the-art likelihood on the binarized OMNIGLOT dataset.
Implicit Generation and Generalization in Energy-Based Models
TLDR
This work presents techniques to scale MCMC based EBM training on continuous neural networks, and shows its success on the high-dimensional data domains of ImageNet32x32, ImageNet128x128, CIFAR-10, and robotic hand trajectories, achieving better samples than other likelihood models and nearing the performance of contemporary GAN approaches.
A Style-Based Generator Architecture for Generative Adversarial Networks
  • Tero Karras, S. Laine, Timo Aila
  • Computer Science, Mathematics
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation.
A Contrastive Learning Approach for Training Variational Autoencoder Priors
TLDR
Noise contrastive priors are proposed that improve the generative performance of state-of-the-art VAEs by a large margin on the MNIST, CIFAR-10, CelebA 64, and CelebA HQ 256 datasets.
Compositional Visual Generation with Energy Based Models
TLDR
This paper shows that energy-based models can exhibit this ability by directly combining probability distributions, and showcases the breadth of unique capabilities of the model, such as the ability to continually learn and incorporate new concepts, or infer compositions of concept properties underlying an image.
DVAE++: Discrete Variational Autoencoders with Overlapping Transformations
TLDR
DVAE++ is developed, a generative model with a global discrete prior and a hierarchy of convolutional continuous variables, and a new variational bound to efficiently train with Boltzmann machine priors is derived.
Analyzing and Improving the Image Quality of StyleGAN
TLDR
This work redesigns the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images, and thereby redefines the state of the art in unconditional image modeling.
Generative Modeling by Estimating Gradients of the Data Distribution
TLDR
A new generative model where samples are produced via Langevin dynamics using gradients of the data distribution estimated with score matching, which allows flexible model architectures, requires no sampling during training or the use of adversarial methods, and provides a learning objective that can be used for principled model comparisons.
Semi-supervised Learning with Deep Generative Models
TLDR
It is shown that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.
On the "steerability" of generative adversarial networks
TLDR
It is shown that although current GANs can fit standard datasets very well, they still fall short of being comprehensive models of the visual manifold, and it is hypothesized that the degree of distributional shift is related to the breadth of the training data distribution.
...
1
2
3
4
5
...