SinGAN: Learning a Generative Model From a Single Natural Image

@article{Shaham2019SinGANLA,
  title={SinGAN: Learning a Generative Model From a Single Natural Image},
  author={Tamar Rott Shaham and Tali Dekel and Tomer Michaeli},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={4569-4579}
}
We introduce SinGAN, an unconditional generative model that can be learned from a single natural image. [] Key Method In contrast to previous single image GAN schemes, our approach is not limited to texture images, and is not conditional (i.e. it generates samples from noise). User studies confirm that the generated samples are commonly confused to be real images. We illustrate the utility of SinGAN in a wide range of image manipulation tasks.
View With Semantic Reader
384 Citations
Highly Influential Citations
67
Background Citations
203
Methods Citations
126
Results Citations
17

384 Citations

Shuffling-SinGAN: Improvement on Generative Model from a Single Image
TLDR
Shuffling-SinGAN allows generating new samples of arbitrary size and aspect ratio, that have significant variability, yet maintain both the global structure and the fine textures of the training image, and has competitive performance on random image generation.
Patchwise Generative ConvNet: Training Energy-Based Models from a Single Natural Image for Internal Learning
TLDR
The proposed model is simple and natural in that it does not require an auxiliary model (e.g., discriminator) to assist the training, and also unifies internal statistics learning and image generation in a single framework.
PetsGAN: Rethinking Priors for Single Image Generation
TLDR
A regularized latent variable model is introduced to SIG for the first time to give a clear formulation and optimization goal of SIG and a novel Prior-based end-to-end training GAN (PetsGAN) is designed to overcome the problems of SinGAN.
ExSinGAN: Learning an Explainable Generative Model from a Single Image
TLDR
This paper proposes a hierarchical framework that simplifies the learning of the intractable conditional distribution through the successivelearning of the distributions about structure, semantics, and texture, making the generative model more comprehen-sible compared with previous works.
Recurrent SinGAN: Towards Scale-Agnostic Single Image GANs
  • Xiaoyu He, Zhenyong Fu
  • Computer Science
    Proceedings of the 2021 5th International Conference on Electronic Information Technology and Computer Engineering
  • 2021
TLDR
This paper proposes a recurrent generator to replace the pyramid of generators in SinGAN with a single recurrent generator, which can learn the patch distributions across multiple scales, yielding a scale-agnostic single image generative model.
Meta Internal Learning
TLDR
A meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively and reduce training time per image, without loss in performance, and introduce novel capabilities, such as interpolation and feedforward modeling of novel images.
Improved Techniques for Training Single-Image GANs
TLDR
This work conducts a number of experiments to understand the challenges of training generative models from a single image, and proposes some best practices that allowed it to generate improved results over previous work.
One-Shot GAN: Learning to Generate Samples from Single Images and Videos
TLDR
This work introduces One-Shot GAN, an unconditional generative model that can learn to generate samples from a single training image or a single video clip, and proposes a twobranch discriminator architecture, with content and layout branches designed to judge internal content and scene layout realism separately from each other.
InGAN: Capturing and Retargeting the “DNA” of a Natural Image
TLDR
An ``Internal GAN'' (InGAN) -- an image-specific GAN -- which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
Generating Novel Scene Compositions from Single Images and Videos
TLDR
This work introduces SIV-GAN, an unconditional generative model that can generate new scene compositions from a single training image or a single video clip, and proposes a two-branch discriminator architecture, with content and layout branches designed to judge internal content and scene layout realism separately from each other.
...
...

References

SHOWING 1-10 OF 75 REFERENCES
Internal Distribution Matching for Natural Image Retargeting
TLDR
A Deep-Learning approach for retargeting, based on an "Internal GAN" (InGAN), an image-specific GAN that incorporates the Internal statistics of a single natural image in a GAN and is able to synthesize natural looking target images composed from the input image patch-distribution.
InGAN: Capturing and Retargeting the “DNA” of a Natural Image
TLDR
An ``Internal GAN'' (InGAN) -- an image-specific GAN -- which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
InGAN: Capturing and Remapping the "DNA" of a Natural Image
TLDR
An "Internal GAN" (InGAN) is proposed - an image-specific GAN - which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
Deep Image Prior
TLDR
It is shown that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, super-resolution, and inpainting.
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
TLDR
A generative parametric model capable of producing high quality samples of natural images using a cascade of convolutional networks within a Laplacian pyramid framework to generate images in a coarse-to-fine fashion.
Large Scale GAN Training for High Fidelity Natural Image Synthesis
TLDR
It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.
Generative Image Inpainting with Contextual Attention
TLDR
This work proposes a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
Texture Synthesis with Spatial Generative Adversarial Networks
TLDR
This is the first successful completely data-driven texture synthesis method based on GANs, and has the following features which make it a state of the art algorithm for texture synthesis: high image quality of the generated textures, very high scalability w.r.t. the output texture size, fast real-time forward generation.
Generative Visual Manipulation on the Natural Image Manifold
TLDR
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
Learning Texture Manifolds with the Periodic Spatial GAN
TLDR
It is shown that the image generation with PSGANs has properties of a texture manifold: it can smoothly interpolate between samples in the structured noise space and generate novel samples, which lie perceptually between the textures of the original dataset.
...
...