3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning

@article{Bodur20213DDG,
  title={3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning},
  author={Rumeysa Bodur and Binod Bhattarai and Tae-Kyun Kim},
  journal={2021 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  year={2021},
  pages={2391-2400}
}
Manipulating facial expressions is a challenging task due to fine-grained shape changes produced by facial muscles and the lack of input-output pairs for supervised learning. Unlike previous methods using Generative Adversarial Networks (GAN), which rely on cycle-consistency loss or sparse geometry (landmarks) loss for expression synthesis, we propose a novel GAN framework to exploit 3D dense (depth and surface normals) information for expression manipulation. However, a large-scale dataset… 

Figures and Tables from this paper

A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation
TLDR
This work designed a unified architecture of semantic segmentation and hierarchical GANs that is an end-toend differentiable architecture that allows both architectures to benefit from each other.
Face Parsing from RGB and Depth Using Cross-Domain Mutual Learning
TLDR
A novel framework to jointly learn RGB face parsing, depth face parsing and RGB-to-depth domain translation is introduced, which can be effective even when only a small amount of annotated depth data is available for training.
Fast Modelling Algorithm for Realistic Three-Dimensional Human Face for Film and Television Animation
TLDR
A new fast three-dimensional (3D) face modelling algorithm based on the LBF algorithm that has good reconstruction performance and realtime performance and can realize the rapid modelling of human face is proposed.

References

SHOWING 1-10 OF 42 REFERENCES
Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model
TLDR
A novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with wide ranges of expressions, poses, and illuminations conditioned by a 3D morphable model achieves comparable results to the state-of-the-art.
Geometry Guided Adversarial Facial Expression Synthesis
TLDR
A Geometry-Guided Generative Adversarial Network (G2-GAN) for continuously-adjusting and identity-preserving facial expression synthesis and can generate compelling perceptual results on different expression editing tasks.
GANimation: Anatomically-aware Facial Animation from a Single Image
TLDR
A novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression, and proposes a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs.
GAGAN: Geometry-Aware Generative Adversarial Networks
TLDR
Experimental results on face generation indicate that the GAGAN can generate realistic images of faces with arbitrary facial attributes such as facial expression, pose, and morphology, that are of better quality than current GAN-based methods.
GP-GAN: Gender Preserving GAN for Synthesizing Faces from Landmarks
TLDR
A novel face-synthesis method known as Gender Preserving Generative Adversarial Network (GP-GAN) that is guided by adversarial loss, perceptual loss and a gender preserving loss is presented and a novel generator sub-network UDeNet for GP-GAN that leverages advantages of U-Net and DenseNet architectures is proposed.
On the Benefit of Adversarial Training for Monocular Depth Estimation
Geometry-Contrastive GAN for Facial Expression Transfer.
TLDR
Experimental results demonstrate that the proposed Geometry-Contrastive Generative Adversarial Network can be applied in facial expression transfer even there exist big differences in facial shapes and expressions between different subjects.
Neural Face Editing with Intrinsic Image Disentangling
TLDR
An end-to-end generative adversarial network is proposed that infers a face-specific disentangled representation of intrinsic face properties, including shape, albedo, and lighting, and an alpha matte, and it is shown that this network can be trained on in thewild images by incorporating an in-network physically-based image formation module and appropriate loss functions.
Triple consistency loss for pairing distributions in GAN-based face synthesis
TLDR
This work incorporates the triple consistency loss into the training of a new landmark-guided face to face synthesis, where, contrary to previous works, the generated images can simultaneously undergo a large transformation in both expression and pose.
Extreme 3D Face Reconstruction: Seeing Through Occlusions
TLDR
A layered approach which decouples estimation of a global shape from its mid-level details (e.g., wrinkles) and then separately layer this foundation with details represented by a bump map, motivated by the concept of bump mapping is proposed.
...
...