• Corpus ID: 220347627

Collaborative Learning for Faster StyleGAN Embedding

@article{Guan2020CollaborativeLF,
  title={Collaborative Learning for Faster StyleGAN Embedding},
  author={Shanyan Guan and Ying Tai and Bingbing Ni and Feida Zhu and Feiyue Huang and Xiaokang Yang},
  journal={ArXiv},
  year={2020},
  volume={abs/2007.01758}
}
The latent code of the recent popular model StyleGAN has learned disentangled representations thanks to the multi-layer style-based generator. Embedding a given image back to the latent space of StyleGAN enables wide interesting semantic image editing applications. Although previous works are able to yield impressive inversion results based on an optimization framework, which however suffers from the efficiency issue. In this work, we propose a novel collaborative learning framework that… 
Improved StyleGAN Embedding: Where are the Good Latents?
TLDR
A new normalized space is introduced to analyze the diversity and the quality of the reconstructed latent codes and can help answer the question of where good latent codes are located in latent space.
Expanding the Latent Space of StyleGAN for Real Face Editing
TLDR
A method to expand the latent space of StyleGAN with additional content features to break down the trade-off between low-distortion and high-editability and proposes a two-branch model, where the style branch first tackles the entanglement issue by the sparse manipulation of latent codes, and the content branch then mitigates the distortion issue by leveraging the content and appearance details from the input image.
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
TLDR
This paper presents a new feed-forward network “E2Style” for StyleGAN inversion, with significant improvement in terms of efficiency and effectiveness, and introduces a shallower backbone with multiple efficient heads across scales.
Inversion Based on a Detached Dual-Channel Domain Method for StyleGAN2 Embedding
TLDR
The qualitative and quantitative experimental results show that the well-trained encoder can embed an image into a latent code in StyleGAN2 latent space with less time than its peers while preserving facial identity and image details well.
Overparameterization Improves StyleGAN Inversion
TLDR
This work directly and dramatically overparameterize the latent space, before training, with simple changes to the original StyleGAN architecture, and shows that this allows us to obtain near-perfect image reconstruction without the need for encoders nor for altering the latentspace after training.
Semantic Unfolding of StyleGAN Latent Space
TLDR
This paper identifies that the facial attribute disentanglement is not optimal, thus facial editing relying on linear attribute separation is poor, and proposes to improve semantic disentangled with supervision, which leads to a more efficient space for face image editing.
ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement
TLDR
This work presents a novel inversion scheme that extends current encoder-based inversion methods by introducing an iterative refinement mechanism, and presents a residual-based encoder, named ReStyle, which attains improved accuracy compared to current state-of-the-art encoders with a negligible increase in inference time.
Cycle Encoding of a StyleGAN Encoder for Improved Reconstruction and Editability
TLDR
It is shown that both reconstruction and editability can be improved by a proper design of the pivot code, and a simple yet effec-tive method, named cycle encoding, is presented, for a high-quality pivot code.
InvGAN: Invertable GANs
TLDR
This work proposes a general framework that is agnostic to architecture and datasets and successfully embeds real images to the latent space of a high quality generative model, allowing it to perform image inpainting, merging, interpolation and online data augmentation.
LatentKeypointGAN: Controlling GANs via Latent Keypoints
TLDR
LatentKeypointGAN is introduced, a two-stage GAN that is trained endto-end on the classical GAN objective yet internally conditioned on a set of sparse keypoints with associated appearance embeddings that respectively control the position and style of the generated objects and their parts.
...
...

References

SHOWING 1-10 OF 41 REFERENCES
Analyzing and Improving the Image Quality of StyleGAN
TLDR
This work redesigns the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images, and thereby redefines the state of the art in unconditional image modeling.
Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
We propose an efficient algorithm to embed a given image into the latent space of StyleGAN. This embedding enables semantic image editing operations that can be applied to existing photographs.
Interpreting the Latent Space of GANs for Semantic Face Editing
TLDR
This work proposes a novel framework, called InterFaceGAN, for semantic face editing by interpreting the latent semantics learned by GANs, and finds that the latent code of well-trained generative models actually learns a disentangled representation after linear transformations.
A Style-Based Generator Architecture for Generative Adversarial Networks
TLDR
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation.
Toward Multimodal Image-to-Image Translation
TLDR
This work aims to model a distribution of possible outputs in a conditional generative modeling setting that helps prevent a many-to-one mapping from the latent code to the output during training, also known as the problem of mode collapse.
Image2StyleGAN++: How to Edit the Embedded Images?
TLDR
A framework that combines embedding with activation tensor manipulation to perform high quality local edits along with global semantic edits on images and can restore high frequency features in images and thus significantly improves the quality of reconstructed images.
Instance Normalization: The Missing Ingredient for Fast Stylization
TLDR
A small change in the stylization architecture results in a significant qualitative improvement in the generated images, and can be used to train high-performance architectures for real-time image generation.
Inverting the Generator of a Generative Adversarial Network
TLDR
This paper introduces a technique, inversion, to project data samples, specifically images, to the latent space using a pretrained GAN, and demonstrates how the proposed inversion technique may be used to quantitatively compare the performance of various GAN models trained on three image data sets.
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
TLDR
This paper presents a simple yet effective approach that for the first time enables arbitrary style transfer in real-time, comparable to the fastest existing approach, without the restriction to a pre-defined set of styles.
Large Scale GAN Training for High Fidelity Natural Image Synthesis
TLDR
It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.
...
...