Semantic Palette: Guiding Scene Generation with Class Proportions

@article{Moing2021SemanticPG,
  title={Semantic Palette: Guiding Scene Generation with Class Proportions},
  author={Guillaume Le Moing and Tuan-Hung Vu and Himalaya Jain and Patrick P'erez and Matthieu Cord},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={9338-9346}
}
  • G. L. Moing, Tuan-Hung Vu, +2 authors M. Cord
  • Published 1 June 2021
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Despite the recent progress of generative adversarial networks (GANs) at synthesizing photo-realistic images, producing complex urban scenes remains a challenging problem. Previous works break down scene generation into two consecutive phases: unconditional semantic layout synthesis and image synthesis conditioned on layouts. In this work, we propose to condition layout generation as well for higher semantic control: given a vector of class proportions, we generate layouts with matching… Expand

References

SHOWING 1-10 OF 39 REFERENCES
Semantic Bottleneck Scene Generation
TLDR
This work proposes a semantic bottleneck GAN model for unconditional synthesis of complex scenes that outperforms state-of-the-art generative models in unsupervised image synthesis on two challenging domains in terms of the Frechet Inception Distance and user-study evaluations. Expand
Decomposing Image Generation into Layout Prediction and Conditional Synthesis
TLDR
This article investigates splitting the optimisation of generative adversarial networks into two parts, by first generating a semantic segmentation masks from noise and then translating that segmentation mask into an image. Expand
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
TLDR
A new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs) is presented, which significantly outperforms existing methods, advancing both the quality and the resolution of deep image synthesis and editing. Expand
Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
TLDR
This work argues that convolutional kernels in the generator should be aware of the distinct semantic labels at different locations when generating images, and proposes a feature pyramid semantics-embedding discriminator, which is more effective in enhancing fine details and semantic alignments between the generated images and the input semantic layouts. Expand
Analyzing and Improving the Image Quality of StyleGAN
TLDR
This work redesigns the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images, and thereby redefines the state of the art in unconditional image modeling. Expand
Seeing What a GAN Cannot Generate
TLDR
This work visualize mode collapse at both the distribution level and the instance level, and deploys a semantic segmentation network to compare the distribution of segmented objects in the generated images with the target distribution in the training set. Expand
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
TLDR
This work proposes two novel, complementary methods using (i) entropy loss and (ii) adversarial loss respectively for unsupervised domain adaptation in semantic segmentation with losses based on the entropy of the pixel-wise predictions. Expand
Semantic Image Synthesis With Spatially-Adaptive Normalization
TLDR
S spatially-adaptive normalization is proposed, a simple but effective layer for synthesizing photorealistic images given an input semantic layout that allows users to easily control the style and content of image synthesis results as well as create multi-modal results. Expand
A Style-Based Generator Architecture for Generative Adversarial Networks
TLDR
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. Expand
Image-to-Image Translation with Conditional Adversarial Networks
TLDR
Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Expand
...
1
2
3
4
...