Analyzing and Improving the Image Quality of StyleGAN

@article{Karras2020AnalyzingAI,
  title={Analyzing and Improving the Image Quality of StyleGAN},
  author={Tero Karras and Samuli Laine and Miika Aittala and Janne Hellsten and Jaakko Lehtinen and Timo Aila},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={8107-8116}
}
  • Tero Karras, S. Laine, +3 authors Timo Aila
  • Published 3 December 2019
  • Computer Science, Engineering, Mathematics
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images. In addition to improving image… 
SWAGAN: A Style-based Wavelet-driven Generative Model
TLDR
A novel general-purpose Style and WAvelet based GAN (SWAGAN) that implements progressive generation in the frequency domain that retains the qualities that allow StyleGAN to serve as a basis for a multitude of editing tasks and induces improved downstream visual quality.
One Model to Reconstruct Them All: A Novel Way to Use the Stochastic Noise in StyleGAN
TLDR
A novel StyleGAN-based autoencoder architecture, which can reconstruct images with very high quality across several data domains, and demonstrates a previously unknown grade of generalizablility by training the encoder and decoder independently and on different datasets.
Styleformer: Transformer based Generative Adversarial Networks with Style Vector
TLDR
This paper explains how a transformer can generate high-quality images, overcoming the disadvantage that convolution operations are difficult to capture global features in an image, and proposes Styleformer, which is a style-based generator for GAN architecture, but a convolution-free transformer- based generator.
StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN
TLDR
This work presents a novel approach to the video synthesis problem that helps to greatly improve visual quality and drastically reduce the amount of training data and resources necessary for generating videos.
StyleSwin: Transformer-based GAN for High-resolution Image Generation
  • Bowen Zhang, Shuyang Gu, +5 authors Baining Guo
  • Computer Science
    ArXiv
  • 2021
TLDR
The StyleSwin, without complex training strategies, excels over StyleGAN on CelebA-HQ 1024, and achieves on-par performance on FFHQ-1024, proving the promise of using transformers for high-resolution image generation.
InvGAN: Invertable GANs
TLDR
This work proposes a general framework that is agnostic to architecture and datasets and successfully embeds real images to the latent space of a high quality generative model, allowing it to perform image inpainting, merging, interpolation and online data augmentation.
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution
TLDR
This work shows that pre-trained Generative Adversarial Networks (GANs), e.g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR) and shows clear improvements in terms of fidelity and texture faithfulness in comparison to existing methods.
Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation
TLDR
A new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space in which both intra- and inter-domain interpolations correspond to gradual changes in the generated images and the content of the source image is better preserved during the translation.
Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks
TLDR
This work proposes an effective approach, termed as LoGAN, to support local editing of the output image of a well-trained GAN model and introduces two operators, i.e., content modulation and style modulation, together with a priority mask to facilitate the precise control of the intermediate generative features.
InvGAN: Invertible GANs
TLDR
This work proposes a general framework that is agnostic to architecture and datasets and successfully embeds real images to the latent space of a high quality generative model, allowing it to perform image inpainting, merging, interpolation and online data augmentation.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 61 REFERENCES
Style Generator Inversion for Image Enhancement and Animation
TLDR
It is shown that differently from earlier GANs, the very recently proposed style-generators are quite easy to invert, and this important observation is used to propose style generators as general purpose image priors.
Progressive Growing of GANs for Improved Quality, Stability, and Variation
TLDR
A new training methodology for generative adversarial networks is described, starting from a low resolution, and adding new layers that model increasingly fine details as training progresses, allowing for images of unprecedented quality.
Large Scale GAN Training for High Fidelity Natural Image Synthesis
TLDR
It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.
A Style-Based Generator Architecture for Generative Adversarial Networks
  • Tero Karras, S. Laine, Timo Aila
  • Computer Science, Mathematics
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation.
Improved Precision and Recall Metric for Assessing Generative Models
TLDR
This work presents an evaluation metric that can separately and reliably measure both the quality and coverage of the samples produced by a generative model and the perceptual quality of individual samples, and extends it to study latent space interpolations.
MSG-GAN: Multi-Scale Gradient GAN for Stable Image Synthesis
TLDR
This work proposes the Multi-Scale Gradient Generative Adversarial Network (MSG-GAN), a simple but effective technique for addressing this problem which allows the flow of gradients from the discriminator to the generator at multiple scales.
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
TLDR
This paper proposes Stacked Generative Adversarial Networks (StackGAN) to generate 256 photo-realistic images conditioned on text descriptions and introduces a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold.
Self-Attention Generative Adversarial Networks
TLDR
The proposed SAGAN achieves the state-of-the-art results, boosting the best published Inception score from 36.8 to 52.52 and reducing Frechet Inception distance from 27.62 to 18.65 on the challenging ImageNet dataset.
On the Effects of Batch and Weight Normalization in Generative Adversarial Networks
Generative adversarial networks (GANs) are highly effective unsupervised learning frameworks that can generate very sharp data, even for data such as images with complex, highly multimodal
Attributing Fake Images to GANs: Analyzing Fingerprints in Generated Images
TLDR
This work asks the question if and to what extend a generated fake image can be attribute to a particular Generative Adversarial Networks (GANs) of a certain architecture and trained with particular data and random seed and shows single samples from GANs carry highly characteristic fingerprints which make attribution of images to GAns possible.
...
1
2
3
4
5
...