SinGAN: Learning a Generative Model From a Single Natural Image

@article{Shaham2019SinGANLA,
  title={SinGAN: Learning a Generative Model From a Single Natural Image},
  author={Tamar Rott Shaham and Tali Dekel and Tomer Michaeli},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={4569-4579}
}
We introduce SinGAN, an unconditional generative model that can be learned from a single natural image. [] Key Method In contrast to previous single image GAN schemes, our approach is not limited to texture images, and is not conditional (i.e. it generates samples from noise). User studies confirm that the generated samples are commonly confused to be real images. We illustrate the utility of SinGAN in a wide range of image manipulation tasks.
View on IEEE
View With Semantic Reader
359 Citations
Highly Influential Citations
65
Background Citations
194
Methods Citations
117
Results Citations
16

359 Citations

Citation Type

Citation Type

Shuffling-SinGAN: Improvement on Generative Model from a Single Image
TLDR
Shuffling-SinGAN allows generating new samples of arbitrary size and aspect ratio, that have significant variability, yet maintain both the global structure and the fine textures of the training image, and has competitive performance on random image generation.
Patchwise Generative ConvNet: Training Energy-Based Models from a Single Natural Image for Internal Learning
TLDR
The proposed model is simple and natural in that it does not require an auxiliary model (e.g., discriminator) to assist the training, and also unifies internal statistics learning and image generation in a single framework.
ExSinGAN: Learning an Explainable Generative Model from a Single Image
TLDR
This paper proposes a hierarchical framework that simplifies the learning of the intractable conditional distribution through the successivelearning of the distributions about structure, semantics, and texture, making the generative model more comprehensible compared with previous works.
PetsGAN: Rethinking Priors for Single Image Generation
TLDR
A regularized latent variable model is introduced to SIG for the first time to give a clear formulation and optimization goal of SIG and a novel Prior-based end-to-end training GAN (PetsGAN) is designed to overcome the problems of SinGAN.
Recurrent SinGAN: Towards Scale-Agnostic Single Image GANs
  • Xiaoyu He, Zhenyong Fu
  • Computer Science
    Proceedings of the 2021 5th International Conference on Electronic Information Technology and Computer Engineering
  • 2021
TLDR
This paper proposes a recurrent generator to replace the pyramid of generators in SinGAN with a single recurrent generator, which can learn the patch distributions across multiple scales, yielding a scale-agnostic single image generative model.
Meta Internal Learning
TLDR
The results show that the models obtained are as suitable as single-image GANs for many common image applications, significantly reduce the training time per image without loss in performance, and introduce novel capabilities, such as interpolation and feedforward modeling of novel images.
One-Shot GAN: Learning to Generate Samples from Single Images and Videos
TLDR
This work introduces One-Shot GAN, an unconditional generative model that can learn to generate samples from a single training image or a single video clip, and proposes a twobranch discriminator architecture, with content and layout branches designed to judge internal content and scene layout realism separately from each other.
Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Models
TLDR
This paper revisit and cast the “good-old” patchbased methods into a novel optimization-free framework, which produces superior results, less artifacts and more realistic global structure than any of the previous approaches (whether GAN-based or classical patch-based).
InGAN: Capturing and Retargeting the “DNA” of a Natural Image
TLDR
An ``Internal GAN'' (InGAN) -- an image-specific GAN -- which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
Diverse Single Image Generation with Controllable Global Structure through Self-Attention
TLDR
This work uses adversarial feedback to make the quality of the generation better, and its results are visually better than the state-of-the-art, particularly, in generating images that require global context.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 78 REFERENCES
Internal Distribution Matching for Natural Image Retargeting
TLDR
A Deep-Learning approach for retargeting, based on an "Internal GAN" (InGAN), an image-specific GAN that incorporates the Internal statistics of a single natural image in a GAN and is able to synthesize natural looking target images composed from the input image patch-distribution.
InGAN: Capturing and Retargeting the “DNA” of a Natural Image
TLDR
An ``Internal GAN'' (InGAN) -- an image-specific GAN -- which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
InGAN: Capturing and Remapping the "DNA" of a Natural Image
TLDR
An "Internal GAN" (InGAN) is proposed - an image-specific GAN - which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
Deep Image Prior
TLDR
It is shown that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, superresolution, and inpainting.
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
TLDR
A generative parametric model capable of producing high quality samples of natural images using a cascade of convolutional networks within a Laplacian pyramid framework to generate images in a coarse-to-fine fashion.
Large Scale GAN Training for High Fidelity Natural Image Synthesis
TLDR
It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.
Generative Image Inpainting with Contextual Attention
TLDR
This work proposes a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
Texture Synthesis with Spatial Generative Adversarial Networks
TLDR
This is the first successful completely data-driven texture synthesis method based on GANs, and has the following features which make it a state of the art algorithm for texture synthesis: high image quality of the generated textures, very high scalability w.r.t. the output texture size, fast real-time forward generation.
Generative Visual Manipulation on the Natural Image Manifold
TLDR
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
Learning Texture Manifolds with the Periodic Spatial GAN
TLDR
It is shown that the image generation with PSGANs has properties of a texture manifold: it can smoothly interpolate between samples in the structured noise space and generate novel samples, which lie perceptually between the textures of the original dataset.
...
1
2
3
4
5
...