Semantic Image Synthesis via Adversarial Learning

@article{Dong2017SemanticIS,
  title={Semantic Image Synthesis via Adversarial Learning},
  author={Hao Dong and Simiao Yu and Chao Wu and Yike Guo},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
  year={2017},
  pages={5707-5715}
}
  • Hao Dong, Simiao Yu, Yike Guo
  • Published 21 July 2017
  • Computer Science
  • 2017 IEEE International Conference on Computer Vision (ICCV)
In this paper, we propose a way of synthesizing realistic images directly with natural language description, which has many useful applications, e.g. intelligent image manipulation. We attempt to accomplish such synthesis: given a source image and a target text description, our model synthesizes images to meet two requirements: 1) being realistic while matching the target text description; 2) maintaining other image features that are irrelevant to the text description. The model should be able… 

Figures and Tables from this paper

Semantic Image Synthesis via Conditional Cycle-Generative Adversarial Networks
TLDR
This paper proposes a novel framework called Conditional Cycle-Generative Adversarial Network (CCGAN), which can generate photo-realistic images conditioned on the given text descriptions, while maintaining the attributes of the original images.
Paired-D GAN for Semantic Image Synthesis
TLDR
Experimental results show that Paired-GAN is capable of semantically synthesizing images to match an input text description while retaining the background in a source image against the state-of-the-art methods.
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
TLDR
A new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs) is presented, which significantly outperforms existing methods, advancing both the quality and the resolution of deep image synthesis and editing.
SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks
TLDR
This work proposes SIMGAN, a generative adversarial networks (GAN) based architecture that is capable of generating images of size 256 × 256 for SIM and demonstrates the effectiveness of SIMGAN and its superiority over existing methods via qualitative and quantitative evaluation on Caltech-200 and Oxford-102 datasets.
Progressive Semantic Image Synthesis via Generative Adversarial Network
TLDR
A novel network architecture to progressively manipulate an image starting from low-resolution, while introducing the original image of corresponding size at different stages with the proposed union module to avoid losing of detail is presented.
Adversarial Text-to-Image Synthesis: A Review
Semantics-Enhanced Adversarial Nets for Text-to-Image Synthesis
TLDR
This paper introduces two modules, a Semantic Consistency Module (SCM) and an Attention Competition Module (ACM), to the SEGAN, a new model, Semantics-enhanced Generative Adversarial Network (SEGAN), for fine-grained text-to-image generation.
Semantics Images Synthesis and Resolution Refinement Using Generative Adversarial Networks
TLDR
A method to synthesizing a super-resolution image with the given image and text descriptions and shows that the generated images not only match the given descriptions well but also maintain detailed features of original images with a higher resolution.
Instance Map Based Image Synthesis With a Denoising Generative Adversarial Network
TLDR
This work proposes a novel denoising framework that includes a one-hot semantic label map to force the generator to pay more attention to the generation of overlapping objects in the generation process of a semantic layout-based image synthesizing model.
Instance Mask Embedding and Attribute-Adaptive Generative Adversarial Network for Text-to-Image Synthesis
TLDR
The instance mask embedding and attribute-adaptive generative adversarial network (IMEAA-GAN) is proposed, which provides local refinement generators with the specific training signals to explicitly generate instance-level features.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 42 REFERENCES
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
TLDR
This paper proposes Stacked Generative Adversarial Networks (StackGAN) to generate 256 photo-realistic images conditioned on text descriptions and introduces a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold.
Learning What and Where to Draw
TLDR
This work proposes a new model, the Generative Adversarial What-Where Network (GAWWN), that synthesizes images given instructions describing what content to draw in which location, and shows high-quality 128 x 128 image synthesis on the Caltech-UCSD Birds dataset.
Generative Adversarial Text to Image Synthesis
TLDR
A novel deep architecture and GAN formulation is developed to effectively bridge advances in text and image modeling, translating visual concepts from characters to pixels.
Generative Image Modeling Using Style and Structure Adversarial Networks
TLDR
This paper factorize the image generation process and proposes Style and Structure Generative Adversarial Network, a model that is interpretable, generates more realistic images and can be used to learn unsupervised RGBD representations.
Image-to-Image Translation with Conditional Adversarial Networks
TLDR
Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Generative Visual Manipulation on the Natural Image Manifold
TLDR
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
Improved Techniques for Training GANs
TLDR
This work focuses on two applications of GANs: semi-supervised learning, and the generation of images that humans find visually realistic, and presents ImageNet samples with unprecedented resolution and shows that the methods enable the model to learn recognizable features of ImageNet classes.
Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
TLDR
Markovian Generative Adversarial Networks (MGANs) are proposed, a method for training generative networks for efficient texture synthesis that surpasses previous neural texture synthesizers by a significant margin and applies to texture synthesis, style transfer, and video stylization.
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
TLDR
SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.
Neural Photo Editing with Introspective Adversarial Networks
TLDR
The Neural Photo Editor is presented, an interface that leverages the power of generative neural networks to make large, semantically coherent changes to existing images, and the Introspective Adversarial Network is introduced, a novel hybridization of the VAE and GAN.
...
1
2
3
4
5
...