SinGAN: Learning a Generative Model From a Single Natural Image
@article{Shaham2019SinGANLA, title={SinGAN: Learning a Generative Model From a Single Natural Image}, author={Tamar Rott Shaham and Tali Dekel and Tomer Michaeli}, journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)}, year={2019}, pages={4569-4579} }
We introduce SinGAN, an unconditional generative model that can be learned from a single natural image. [] Key Method In contrast to previous single image GAN schemes, our approach is not limited to texture images, and is not conditional (i.e. it generates samples from noise). User studies confirm that the generated samples are commonly confused to be real images. We illustrate the utility of SinGAN in a wide range of image manipulation tasks.
Figures and Tables from this paper
384 Citations
Shuffling-SinGAN: Improvement on Generative Model from a Single Image
- Computer ScienceJournal of Physics: Conference Series
- 2021
Shuffling-SinGAN allows generating new samples of arbitrary size and aspect ratio, that have significant variability, yet maintain both the global structure and the fine textures of the training image, and has competitive performance on random image generation.
Patchwise Generative ConvNet: Training Energy-Based Models from a Single Natural Image for Internal Learning
- Computer Science2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2021
The proposed model is simple and natural in that it does not require an auxiliary model (e.g., discriminator) to assist the training, and also unifies internal statistics learning and image generation in a single framework.
PetsGAN: Rethinking Priors for Single Image Generation
- Computer ScienceArXiv
- 2022
A regularized latent variable model is introduced to SIG for the first time to give a clear formulation and optimization goal of SIG and a novel Prior-based end-to-end training GAN (PetsGAN) is designed to overcome the problems of SinGAN.
ExSinGAN: Learning an Explainable Generative Model from a Single Image
- Computer ScienceBMVC
- 2021
This paper proposes a hierarchical framework that simplifies the learning of the intractable conditional distribution through the successivelearning of the distributions about structure, semantics, and texture, making the generative model more comprehen-sible compared with previous works.
Recurrent SinGAN: Towards Scale-Agnostic Single Image GANs
- Computer ScienceProceedings of the 2021 5th International Conference on Electronic Information Technology and Computer Engineering
- 2021
This paper proposes a recurrent generator to replace the pyramid of generators in SinGAN with a single recurrent generator, which can learn the patch distributions across multiple scales, yielding a scale-agnostic single image generative model.
Meta Internal Learning
- Computer ScienceNeurIPS
- 2021
A meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively and reduce training time per image, without loss in performance, and introduce novel capabilities, such as interpolation and feedforward modeling of novel images.
Improved Techniques for Training Single-Image GANs
- Computer Science2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
- 2021
This work conducts a number of experiments to understand the challenges of training generative models from a single image, and proposes some best practices that allowed it to generate improved results over previous work.
One-Shot GAN: Learning to Generate Samples from Single Images and Videos
- Computer Science2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- 2021
This work introduces One-Shot GAN, an unconditional generative model that can learn to generate samples from a single training image or a single video clip, and proposes a twobranch discriminator architecture, with content and layout branches designed to judge internal content and scene layout realism separately from each other.
InGAN: Capturing and Retargeting the “DNA” of a Natural Image
- Computer Science2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- 2019
An ``Internal GAN'' (InGAN) -- an image-specific GAN -- which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
Generating Novel Scene Compositions from Single Images and Videos
- Computer Science
- 2021
This work introduces SIV-GAN, an unconditional generative model that can generate new scene compositions from a single training image or a single video clip, and proposes a two-branch discriminator architecture, with content and layout branches designed to judge internal content and scene layout realism separately from each other.
References
SHOWING 1-10 OF 75 REFERENCES
Internal Distribution Matching for Natural Image Retargeting
- Computer ScienceArXiv
- 2018
A Deep-Learning approach for retargeting, based on an "Internal GAN" (InGAN), an image-specific GAN that incorporates the Internal statistics of a single natural image in a GAN and is able to synthesize natural looking target images composed from the input image patch-distribution.
InGAN: Capturing and Retargeting the “DNA” of a Natural Image
- Computer Science2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- 2019
An ``Internal GAN'' (InGAN) -- an image-specific GAN -- which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
InGAN: Capturing and Remapping the "DNA" of a Natural Image
- Computer Science
- 2018
An "Internal GAN" (InGAN) is proposed - an image-specific GAN - which trains on a single input image and learns its internal distribution of patches, which provides a unified framework for a variety of tasks, bridging the gap between textures and natural images.
Deep Image Prior
- Computer Science2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- 2018
It is shown that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, super-resolution, and inpainting.
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- Computer ScienceNIPS
- 2015
A generative parametric model capable of producing high quality samples of natural images using a cascade of convolutional networks within a Laplacian pyramid framework to generate images in a coarse-to-fine fashion.
Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Computer ScienceICLR
- 2019
It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.
Generative Image Inpainting with Contextual Attention
- Computer Science2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- 2018
This work proposes a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
Texture Synthesis with Spatial Generative Adversarial Networks
- Computer ScienceArXiv
- 2016
This is the first successful completely data-driven texture synthesis method based on GANs, and has the following features which make it a state of the art algorithm for texture synthesis: high image quality of the generated textures, very high scalability w.r.t. the output texture size, fast real-time forward generation.
Generative Visual Manipulation on the Natural Image Manifold
- Computer Science, ArtECCV
- 2016
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
Learning Texture Manifolds with the Periodic Spatial GAN
- Computer ScienceICML
- 2017
It is shown that the image generation with PSGANs has properties of a texture manifold: it can smoothly interpolate between samples in the structured noise space and generate novel samples, which lie perceptually between the textures of the original dataset.