High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

@article{Wang2018HighResolutionIS,
  title={High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs},
  author={Ting-Chun Wang and Ming-Yu Liu and Jun-Yan Zhu and Andrew Tao and Jan Kautz and Bryan Catanzaro},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={8798-8807}
}
We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs. [...] Key Method Furthermore, we extend our framework to interactive visual manipulation with two additional features. First, we incorporate object instance segmentation information, which enables object manipulations such as removing/adding objects and changing the object category.Expand
Image generation from bounding box-represented semantic labels
TLDR
A new translator-enhancer framework by combining cGAN and GAN to achieve high quality image generation is proposed, which is used as a translator to match the semantic constraints whereas GAN is employed as an enhancer to provide details and textures.
Mask Embedding in conditional GAN for Guided Synthesis of High Resolution Images
TLDR
This work proposes to use mask embedding mechanism to allow for a more efficient initial feature projection in the generator, and can generate realistic and high resolution facial images up to the resolution of 512*512 with a mask guidance.
SEMANTIC IMAGE SYNTHESIS
Despite their recent successes, GAN models for semantic image synthesis still suffer from poor image quality when trained with only adversarial supervision. Historically, additionally employing the
Semantic Image Synthesis with Trilateral Generative Adversarial Networks
We present a new method that can synthesize photorealistic images from semantic label maps using GANs with a trilateral generator. Due to the limitation of network capacity, it is difficult for scene
Semantic Image Synthesis Manipulation for Stability Problem using Generative Adversarial Networks: A Survey
TLDR
This survey discussed the Generative Adversarial Networks (GANs) model because of the ability to synthesize good samples directly and a literature discussion between different methods used to improve the result of GAN have been discussed which aims to produce better results and generate more samples.
Semanticgan: Generative Adversarial Networks For Semantic Image To Photo-Realistic Image Translation
TLDR
A SemanticGAN to synthesize high resolution image with fine details and realistic textures from the semantic label map and a Semantic Information Preserved Loss (SIPL) to maintain semantic information in the process of the generation via a segmentation model are proposed.
Collaging Class-specific GANs for Semantic Image Synthesis
TLDR
Experiments show that this new approach for high resolution semantic image synthesis can generate high quality images in high resolution while having flexibility of object-level control by using class-specific generators.
Decomposing Image Generation into Layout Prediction and Conditional Synthesis
TLDR
This article investigates splitting the optimisation of generative adversarial networks into two parts, by first generating a semantic segmentation masks from noise and then translating that segmentation mask into an image.
Resolution enhancement in the recovery of underdrawings via style transfer by generative adversarial deep neural networks
TLDR
This work applies generative adversarial convolutional neural networks to the problem of style transfer to underdrawings and ghost-images in x-rays of fine art paintings with a special focus on enhancing their spatial resolution, and shows that even just a few examples of human-generated image segmentations can greatly improve the generated images.
Semantic Controllable Image Generation in Few-shot Settings
Semantic Controllable Image Generation in Few-shot Settings Jianjin Xu Generative Adversarial Networks (GANs) are able to generate high-quality images, but it remains difficult to explicitly specify
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 76 REFERENCES
Semantic Image Synthesis via Adversarial Learning
TLDR
An end-to-end neural architecture that leverages adversarial learning to automatically learn implicit loss functions, which are optimized to fulfill the aforementioned two requirements of being realistic while matching the target text description.
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
TLDR
This paper proposes Stacked Generative Adversarial Networks (StackGAN) to generate 256 photo-realistic images conditioned on text descriptions and introduces a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold.
Scribbler: Controlling Deep Image Synthesis with Sketch and Color
TLDR
A deep adversarial image synthesis architecture that is conditioned on sketched boundaries and sparse color strokes to generate realistic cars, bedrooms, or faces is proposed and demonstrates a sketch based image synthesis system which allows users to scribble over the sketch to indicate preferred color for objects.
Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
TLDR
A novel deep conditional generative adversarial network architecture that takes its strength from the semantic layout and scene attributes integrated as conditioning variables and is able to generate realistic outdoor scene images under different conditions, e.g. day-night, sunny-foggy, with clear object boundaries.
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
  • C. Ledig, Lucas Theis, +6 authors W. Shi
  • Computer Science, Mathematics
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
TLDR
SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.
Photographic Image Synthesis with Cascaded Refinement Networks
  • Qifeng Chen, V. Koltun
  • Computer Science
    2017 IEEE International Conference on Computer Vision (ICCV)
  • 2017
TLDR
It is shown that photographic images can be synthesized from semantic layouts by a single feedforward network with appropriate structure, trained end-to-end with a direct regression objective.
Image-to-Image Translation with Conditional Adversarial Networks
TLDR
Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Generative Image Modeling Using Style and Structure Adversarial Networks
TLDR
This paper factorize the image generation process and proposes Style and Structure Generative Adversarial Network, a model that is interpretable, generates more realistic images and can be used to learn unsupervised RGBD representations.
Generative Visual Manipulation on the Natural Image Manifold
TLDR
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
Progressive Growing of GANs for Improved Quality, Stability, and Variation
TLDR
A new training methodology for generative adversarial networks is described, starting from a low resolution, and adding new layers that model increasingly fine details as training progresses, allowing for images of unprecedented quality.
...
1
2
3
4
5
...