Interpreting the Latent Space of GANs for Semantic Face Editing

@article{Shen2020InterpretingTL,
  title={Interpreting the Latent Space of GANs for Semantic Face Editing},
  author={Yujun Shen and Jinjin Gu and Xiaoou Tang and Bolei Zhou},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={9240-9249}
}
  • Yujun Shen, Jinjin Gu, +1 author Bolei Zhou
  • Published 2020
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Despite the recent advance of Generative Adversarial Networks (GANs) in high-fidelity image synthesis, there lacks enough understandings on how GANs are able to map the latent code sampled from a random distribution to a photo-realistic image. [...] Key Method Based on our analysis, we propose a simple and general technique, called InterFaceGAN, for semantic face editing in latent space.Expand
Latent Space Regression in GANs for Invertible Image Generation
Generative Adversarial Networks (GANs) are one of the most popular generative architectures, yet how the generator convert unstructured latent code into high fidelity output is still an openExpand
LatentKeypointGAN: Controlling GANs via Latent Keypoints
TLDR
LatentKeypointGAN is introduced, a two-stage GAN that is trained endto-end on the classical GAN objective yet internally conditioned on a set of sparse keypoints with associated appearance embeddings that respectively control the position and style of the generated objects and their parts. Expand
Semantic and Geometric Unfolding of StyleGAN Latent Space
TLDR
This paper proposes a new method to learn a proxy latent representation using normalizing flows to remedy limitations of generative adversarial networks, and shows that this leads to a more efficient space for face image editing. Expand
Linear Semantics in Generative Adversarial Networks
TLDR
This work aims to better understand the semantic representation of GANs, and thereby enable semantic control in GAN’s generation process, and finds that a well-trained GAN encodes image semantics in its internal feature maps in a surprisingly simple way. Expand
Mask-Guided Discovery of Semantic Manifolds in Generative Models
TLDR
This work presents a method to explore the manifolds of changes of spatially localized regions of the face and discovers smoothly varying sequences of latent vectors along these manifolds suitable for creating animations. Expand
Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
TLDR
This work shows that highly-structured semantic hierarchy emerges as variation factors from synthesizing scenes from the generative representations in state-of-the-art GAN models, like StyleGAN and BigGAN, and quantifies the causality between the activations and semantics occurring in the output image. Expand
A Latent Transformer for Disentangled Face Editing in Images and Videos
High quality facial image editing is a challenging problem in the movie post-production industry, requiring a high degree of control and identity preservation. Previous works that attempt to tackleExpand
Discovering Interpretable Latent Space Directions of GANs Beyond Binary Attributes
Generative adversarial networks (GANs) learn to map noise latent vectors to high-fidelity image outputs. It is found that the input latent space shows semantic correlations with the output imageExpand
Disentangled Face Attribute Editing via Instance-Aware Latent Space Search
TLDR
A novel framework (IALS) that performs InstanceAware Latent-Space Search to find semantic directions for disentangled attribute editing and outperforms state-of-the-art methods proposed recently by a wide margin. Expand
Transforming Facial Weight of Real Images by Editing Latent Space of StyleGAN
We present an invert-and-edit framework to automatically transform facial weight of an input face image to look thinner or heavier by leveraging semantic facial attributes encoded in the latent spaceExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 53 REFERENCES
Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
TLDR
This work shows that highly-structured semantic hierarchy emerges as variation factors from synthesizing scenes from the generative representations in state-of-the-art GAN models, like StyleGAN and BigGAN, and quantifies the causality between the activations and semantics occurring in the output image. Expand
Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
TLDR
A new algorithm for training generative adversarial networks that jointly learns latent codes for both identities and observations that can generate diverse images of the same subject and traverse the manifold of subjects while maintaining contingent aspects such as lighting and pose. Expand
Disentangled Inference for GANs with Latently Invertible Autoencoder
TLDR
This paper shows that the entanglement of the latent space for the VAE/GAN framework poses the main challenge for encoder learning, and proposes a novel algorithm named Latently Invertible Autoencoder (LIA), which is based on an invertible network and its inverse mapping that is symmetrically embedded in the latentspace of VAE. Expand
Invertible Conditional GANs for image editing
TLDR
This work evaluates encoders to inverse the mapping of a cGAN, i.e., mapping a real image into a latent space and a conditional representation, which allows to reconstruct and modify real images of faces conditioning on arbitrary attributes. Expand
Seeing What a GAN Cannot Generate
TLDR
This work visualize mode collapse at both the distribution level and the instance level, and deploys a semantic segmentation network to compare the distribution of segmented objects in the generated images with the target distribution in the training set. Expand
Image Processing Using Multi-Code GAN Prior
TLDR
A novel approach is proposed, called mGANprior, to incorporate the well-trained GANs as effective prior to a variety of image processing tasks, by employing multiple latent codes to generate multiple feature maps at some intermediate layer of the generator and composing them with adaptive channel importance to recover the input image. Expand
ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes
TLDR
A novel model is proposed which receives two images of opposite attributes as inputs and can transfer exactly the same type of attributes from one image to another by exchanging certain part of their encodings. Expand
A Style-Based Generator Architecture for Generative Adversarial Networks
TLDR
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. Expand
Disentangled Representation Learning GAN for Pose-Invariant Face Recognition
TLDR
Quantitative and qualitative evaluation on both controlled and in-the-wild databases demonstrate the superiority of DR-GAN over the state of the art. Expand
Visualizing and Understanding Generative Adversarial Networks (Extended Abstract)
TLDR
This work presents an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level, and identifies a group of interpretable units that are closely related to concepts with a segmentation-based network dissection method. Expand
...
1
2
3
4
5
...