Analyzing and Improving the Image Quality of StyleGAN

@article{Karras2020AnalyzingAI,
  title={Analyzing and Improving the Image Quality of StyleGAN},
  author={Tero Karras and S. Laine and M. Aittala and Janne Hellsten and J. Lehtinen and Timo Aila},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={8107-8116}
}
  • Tero Karras, S. Laine, +3 authors Timo Aila
  • Published 2020
  • Computer Science, Engineering, Mathematics
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images. In addition to improving image… Expand
SWAGAN: A Style-based Wavelet-driven Generative Model
TLDR
A novel general-purpose Style and WAvelet based GAN (SWAGAN) that implements progressive generation in the frequency domain that retains the qualities that allow StyleGAN to serve as a basis for a multitude of editing tasks and induces improved downstream visual quality. Expand
One Model to Reconstruct Them All: A Novel Way to Use the Stochastic Noise in StyleGAN
TLDR
A novel StyleGAN-based autoencoder architecture, which can reconstruct images with very high quality across several data domains, and demonstrates a previously unknown grade of generalizablility by training the encoder and decoder independently and on different datasets. Expand
StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN
TLDR
This paper presents a novel approach to the video synthesis problem that helps to greatly improve visual quality and drastically reduce the amount of training data and resources necessary for generating video content. Expand
Styleformer: Transformer based Generative Adversarial Networks with Style Vector
TLDR
This paper explains how a transformer can generate high-quality images, overcoming the disadvantage that convolution operations are difficult to capture global features in an image, and proposes Styleformer, which is a style-based generator for GAN architecture, but a convolution-free transformer- based generator. Expand
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution
TLDR
It is shown that pre-trained Generative Adversarial Networks, e.g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR) and shows clear improvements in terms of fidelity and texture faithfulness in comparison to existing methods. Expand
Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation
TLDR
A new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space in which both intraand inter-domain interpolations correspond to gradual changes in the generated images and the content of the source image is better preserved during the translation. Expand
Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks
TLDR
This work proposes an effective approach, termed as LoGAN, to support local editing of the output image of a well-trained GAN model and introduces two operators, i.e., content modulation and style modulation, together with a priority mask to facilitate the precise control of the intermediate generative features. Expand
Designing an encoder for StyleGAN image manipulation
TLDR
This paper carefully study the latent space of StyleGAN, the state-of-the-art unconditional generator, and suggests two principles for designing encoders in a manner that allows one to control the proximity of the inversions to regions that StyleGAN was originally trained on. Expand
Teachers Do More Than Teach: Compressing Image-to-Image Models
TLDR
This work revisits the search space of generative models, introducing an inception-based residual block into generators and proposes a one-step pruning algorithm that searches a student architecture from the teacher model and substantially reduces searching cost. Expand
Fine-Tuning StyleGAN2 For Cartoon Face Generation
TLDR
This paper proposes a novel image-to-image translation method that generates images of the target domain by finetuning a stylegan2 pretrained model, suitable for unsupervised I2I translation on unbalanced datasets. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 61 REFERENCES
Style Generator Inversion for Image Enhancement and Animation
TLDR
It is shown that differently from earlier GANs, the very recently proposed style-generators are quite easy to invert, and this important observation is used to propose style generators as general purpose image priors. Expand
Progressive Growing of GANs for Improved Quality, Stability, and Variation
TLDR
A new training methodology for generative adversarial networks is described, starting from a low resolution, and adding new layers that model increasingly fine details as training progresses, allowing for images of unprecedented quality. Expand
Large Scale GAN Training for High Fidelity Natural Image Synthesis
TLDR
It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input. Expand
A Style-Based Generator Architecture for Generative Adversarial Networks
TLDR
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. Expand
Improved Precision and Recall Metric for Assessing Generative Models
TLDR
This work presents an evaluation metric that can separately and reliably measure both the quality and coverage of the samples produced by a generative model and the perceptual quality of individual samples, and extends it to study latent space interpolations. Expand
MSG-GAN: Multi-Scale Gradient GAN for Stable Image Synthesis
TLDR
This work proposes the Multi-Scale Gradient Generative Adversarial Network (MSG-GAN), a simple but effective technique for addressing this problem which allows the flow of gradients from the discriminator to the generator at multiple scales. Expand
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
TLDR
This paper proposes Stacked Generative Adversarial Networks (StackGAN) to generate 256 photo-realistic images conditioned on text descriptions and introduces a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold. Expand
Self-Attention Generative Adversarial Networks
TLDR
The proposed SAGAN achieves the state-of-the-art results, boosting the best published Inception score from 36.8 to 52.52 and reducing Frechet Inception distance from 27.62 to 18.65 on the challenging ImageNet dataset. Expand
On the Effects of Batch and Weight Normalization in Generative Adversarial Networks
Generative adversarial networks (GANs) are highly effective unsupervised learning frameworks that can generate very sharp data, even for data such as images with complex, highly multimodalExpand
Attributing Fake Images to GANs: Analyzing Fingerprints in Generated Images
TLDR
This work asks the question if and to what extend a generated fake image can be attribute to a particular Generative Adversarial Networks (GANs) of a certain architecture and trained with particular data and random seed and shows single samples from GANs carry highly characteristic fingerprints which make attribution of images to GAns possible. Expand
...
1
2
3
4
5
...