3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning

  title={3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning},
  author={Rumeysa Bodur and Binod Bhattarai and Tae-Kyun Kim},
  journal={2021 IEEE Winter Conference on Applications of Computer Vision (WACV)},
Manipulating facial expressions is a challenging task due to fine-grained shape changes produced by facial muscles and the lack of input-output pairs for supervised learning. Unlike previous methods using Generative Adversarial Networks (GAN), which rely on cycle-consistency loss or sparse geometry (landmarks) loss for expression synthesis, we propose a novel GAN framework to exploit 3D dense (depth and surface normals) information for expression manipulation. However, a large-scale dataset… 

Figures and Tables from this paper

A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation

This work designed a unified architecture of semantic segmentation and hierarchical GANs that is an end-toend differentiable architecture that allows both architectures to benefit from each other.

Fast Modelling Algorithm for Realistic Three-Dimensional Human Face for Film and Television Animation

A new fast three-dimensional (3D) face modelling algorithm based on the LBF algorithm that has good reconstruction performance and realtime performance and can realize the rapid modelling of human face is proposed.

Face Parsing from RGB and Depth Using Cross-Domain Mutual Learning

A novel framework to jointly learn RGB face parsing, depth face parsing and RGB-to-depth domain translation is introduced, which can be effective even when only a small amount of annotated depth data is available for training.



Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model

A novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with wide ranges of expressions, poses, and illuminations conditioned by a 3D morphable model achieves comparable results to the state-of-the-art.

Geometry Guided Adversarial Facial Expression Synthesis

A Geometry-Guided Generative Adversarial Network (G2-GAN) for continuously-adjusting and identity-preserving facial expression synthesis and can generate compelling perceptual results on different expression editing tasks.

GANimation: Anatomically-aware Facial Animation from a Single Image

A novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression, and proposes a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs.

GAGAN: Geometry-Aware Generative Adversarial Networks

Experimental results on face generation indicate that the GAGAN can generate realistic images of faces with arbitrary facial attributes such as facial expression, pose, and morphology, that are of better quality than current GAN-based methods.

GP-GAN: Gender Preserving GAN for Synthesizing Faces from Landmarks

A novel face-synthesis method known as Gender Preserving Generative Adversarial Network (GP-GAN) that is guided by adversarial loss, perceptual loss and a gender preserving loss is presented and a novel generator sub-network UDeNet for GP-GAN that leverages advantages of U-Net and DenseNet architectures is proposed.

On the Benefit of Adversarial Training for Monocular Depth Estimation

Geometry-Contrastive GAN for Facial Expression Transfer.

Experimental results demonstrate that the proposed Geometry-Contrastive Generative Adversarial Network can be applied in facial expression transfer even there exist big differences in facial shapes and expressions between different subjects.

Neural Face Editing with Intrinsic Image Disentangling

An end-to-end generative adversarial network is proposed that infers a face-specific disentangled representation of intrinsic face properties, including shape, albedo, and lighting, and an alpha matte, and it is shown that this network can be trained on in thewild images by incorporating an in-network physically-based image formation module and appropriate loss functions.

Triple consistency loss for pairing distributions in GAN-based face synthesis

This work incorporates the triple consistency loss into the training of a new landmark-guided face to face synthesis, where, contrary to previous works, the generated images can simultaneously undergo a large transformation in both expression and pose.

Extreme 3D Face Reconstruction: Seeing Through Occlusions

A layered approach which decouples estimation of a global shape from its mid-level details (e.g., wrinkles) and then separately layer this foundation with details represented by a bump map, motivated by the concept of bump mapping is proposed.