Image Shape Manipulation from a Single Augmented Training Sample

@article{Vinker2021ImageSM,
  title={Image Shape Manipulation from a Single Augmented Training Sample},
  author={Yael Vinker and Eli K. Horwitz and Nir Zabari and Yedid Hoshen},
  journal={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021},
  pages={13749-13758}
}
In this paper, we present DeepSIM, a generative model for conditional image manipulation based on a single image. We find that extensive augmentation is key for enabling single image training, and incorporate the use of thin-plate-spline (TPS) as an effective augmentation. Our network learns to map between a primitive representation of the image to the image itself. The choice of a primitive representation has an impact on the ease and expressiveness of the manipulations and can be automatic (e… 
ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions
TLDR
Within specific image regions hampering long-range interactions, the novel attention mechanism is both computationally efficient and effective, leading to synthesizing interesting phenomena in scenes that were not possible to generate reliably with previous convnets and transformer approaches.
EpiGRAF: Rethinking training of 3D GANs
TLDR
This work shows that it is possible to obtain a high-resolution 3D generator with SotA image quality by following a completely different route of simply training the model patch-wise, and designs a locationand scale-aware discriminator to work on patches of different proportions and spatial positions.

References

SHOWING 1-10 OF 50 REFERENCES
Generative Visual Manipulation on the Natural Image Manifold
TLDR
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
SinGAN: Learning a Generative Model From a Single Natural Image
We introduce SinGAN, an unconditional generative model that can be learned from a single natural image. Our model is trained to capture the internal distribution of patches within the image, and is
Interactive video stylization using few-shot patch-based training
TLDR
This paper demonstrates how to train an appearance translation network from scratch using only a few stylized exemplars while implicitly preserving temporal consistency and leads to a video stylization framework that supports real-time inference, parallel processing, and random access to an arbitrary output frame.
SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis
  • W. ChenJames Hays
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
TLDR
This work proposes a novel Generative Adversarial Network approach that synthesizes plausible images from 50 categories including motorcycles, horses and couches and introduces a new network building block suitable for both the generator and discriminator which improves the information flow by injecting the input image at multiple scales.
Internal Distribution Matching for Natural Image Retargeting
TLDR
A Deep-Learning approach for retargeting, based on an "Internal GAN" (InGAN), an image-specific GAN that incorporates the Internal statistics of a single natural image in a GAN and is able to synthesize natural looking target images composed from the input image patch-distribution.
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
TLDR
A new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs) is presented, which significantly outperforms existing methods, advancing both the quality and the resolution of deep image synthesis and editing.
Reference-Based Sketch Image Colorization Using Augmented-Self Reference and Dense Semantic Correspondence
TLDR
This paper proposes to utilize the identical image with geometric distortion as a virtual reference, which makes it possible to secure the ground truth for a colored output image.
Approximate Thin Plate Spline Mappings
TLDR
A third approximation method based on a classic matrix completion technique that allows for principal warp analysis as a by-product is described and a significant improvement over the naive method is demonstrated.
Deep Image Prior
TLDR
It is shown that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, super-resolution, and inpainting.
Texture Synthesis with Spatial Generative Adversarial Networks
TLDR
This is the first successful completely data-driven texture synthesis method based on GANs, and has the following features which make it a state of the art algorithm for texture synthesis: high image quality of the generated textures, very high scalability w.r.t. the output texture size, fast real-time forward generation.
...
...