• Corpus ID: 232478397

In&Out : Diverse Image Outpainting via GAN Inversion

@article{Cheng2021InOutD,
  title={In\&Out : Diverse Image Outpainting via GAN Inversion},
  author={Yen-Chi Cheng and Chieh Hubert Lin and Hsin-Ying Lee and Jian Ren and S. Tulyakov and Ming-Hsuan Yang},
  journal={ArXiv},
  year={2021},
  volume={abs/2104.00675}
}
Image outpainting seeks for a semantically consistent extension of the input image beyond its available content. Compared to inpainting — filling in missing pixels in a way coherent with the neighboring pixels — outpainting can be achieved in more diverse ways since the problem is less constrained by the surrounding pixels. Existing image outpainting methods pose the problem as a conditional image-to-image translation task, often generating repetitive structures and textures by replicating the… 
View PDF on arXiv
View With Semantic Reader
12 Citations
Background Citations
8
Methods Citations
5
Results Citations
1

12 Citations

Citation Type

Citation Type

Outpainting Natural Scenery Images by Fusing Forecasting Information
TLDR
A novel Multi-view Recurrent Content Transfer module is embedded into an Encoder-Decoder architecture for long-range all-side image outpainting and a multi-head attention mechanism is leveraged to fuse information from different representation sub-spaces at different positions to enhance the consistency of generated images and the original input.
MaskGIT: Masked Generative Image Transformer
TLDR
This paper proposes a novel image synthesis paradigm using a bidirectional transformer decoder, which it is demonstrated that MaskGIT significantly outperforms the state-of-the-art transformer model on the ImageNet dataset, and accelerates autoregressive decoding by up to 64x.
InfinityGAN: Towards Infinite-Pixel Image Synthesis
TLDR
Experimental evaluation validates that InfinityGAN generates images with superior realism compared to baselines and features parallelizable inference, and several applications unlocked by the approach are shown, such as spatial style fusion, multimodal outpainting, and image inbetweening.
InfinityGAN: Towards Infinite-Resolution Image Synthesis
TLDR
Experimental evaluation supports that InfinityGAN generates images with superior global structure compared to baselines at the same time featuring parallelizable inference, and shows several applications unlocked by the approach, such as fusing styles spatially, multi-modal outpainting and image inbetweening at arbitrary input and output resolutions.
Guided Co-Modulated GAN for 360° Field of View Extrapolation
TLDR
This work proposes a method to extrapolate a 360 ◦ field of view from a single image that allows for user-controlled synthesis of the out-painted content and introduces a novel guided co-modulation framework, which drives the image generation process with a common pretrained discriminative model.
Palette: Image-to-Image Diffusion Models
TLDR
A unified framework for image-to-image translation based on conditional diffusion models is developed and it is shown that a generalist, multi-task diffusion model performs as well or better than task-specific specialist counterparts.
Guided Co-Modulated GAN for 360{\deg} Field of View Extrapolation
TLDR
This work proposes a method to extrapolate a 360 ◦ field of view from a single image that allows for user-controlled synthesis of the out-painted content and introduces a novel guided co-modulation framework, which drives the image generation process with a common pretrained discriminative model.
GAN Inversion: A Survey
TLDR
This paper provides a survey of GAN inversion with a focus on its representative algorithms and its applications in image restoration and image manipulation, and discusses the trends and challenges for future research.
Arbitrary-Scale Image Synthesis
TLDR
This work proposes the design of scale-consistent positional encodings invariant to the generator’s layers transformations that enables the generation of arbitraryscale images even at scales unseen during training.
Any-resolution Training for High-resolution Image Synthesis
TLDR
This work introduces continuous-scale training, a process that samples patches at random scales to train a new generator with variable output resolutions, and proves that every pixel matters and creates datasets with variable-size images, collected at their native resolutions.
...
...

References

SHOWING 1-10 OF 47 REFERENCES
Free-Form Image Inpainting With Gated Convolution
TLDR
The proposed gated convolution solves the issue of vanilla convolution that treats all input pixels as valid ones, generalizes partial convolution by providing a learnable dynamic feature selection mechanism for each channel at each spatial location across all layers.
Generative Image Inpainting with Contextual Attention
TLDR
This work proposes a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
In-Domain GAN Inversion for Real Image Editing
TLDR
An in-domain GAN inversion approach, which not only faithfully reconstructs the input image but also ensures the inverted code to be semantically meaningful for editing, which achieves satisfying real image reconstruction and facilitates various image editing tasks, significantly outperforming start-of-the-arts.
Very Long Natural Scenery Image Prediction by Outpainting
TLDR
This work devise some innovative modules, named Skip Horizontal Connection and Recurrent Content Transfer, and integrate them into their designed encoder-decoder structure, and shows that this network can generate highly realistic outpainting prediction effectively and efficiently.
Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis
TLDR
This work proposes a simple yet effective regularization term to address the mode collapse issue for cGANs and explicitly maximizes the ratio of the distance between generated images with respect to the corresponding latent codes, thus encouraging the generators to explore more minor modes during training.
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
TLDR
This paper proposes Stacked Generative Adversarial Networks (StackGAN) to generate 256 photo-realistic images conditioned on text descriptions and introduces a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold.
Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space
TLDR
This work shows that, under a simple nonlinear operation, the data distribution can be modeled as Gaussian and therefore expressed using sufficient statistics and yields a simple Gaussian prior, which is used to regularize the projection of images into the latent space.
Boundless: Generative Adversarial Networks for Image Extension
TLDR
This work introduces semantic conditioning to the discriminator of a generative adversarial network (GAN) and achieves strong results on image extension with coherent semantics and visually pleasing colors and textures.
Semantic photo manipulation with a generative image prior
TLDR
This paper adapts the image prior learned by GANs to image statistics of an individual image and can accurately reconstruct the input image and synthesize new content, consistent with the appearance of theinput image.
Generative Visual Manipulation on the Natural Image Manifold
TLDR
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
...
...