A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces

@article{Ferrari2020ASA,
  title={A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces},
  author={Claudio Ferrari and Stefano Berretti and Pietro Pala and A. Bimbo},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2020},
  volume={44},
  pages={6667-6682}
}
The 3D Morphable Model (3DMM) is a powerful statistical tool for representing 3D face shapes. To build a 3DMM, a training set of face scans in full point-to-point correspondence is required, and its modeling capabilities directly depend on the variability contained in the training data. Thus, to increase the descriptive power of the 3DMM, establishing a dense correspondence across heterogeneous scans with sufficient diversity in terms of identities, ethnicities, or expressions becomes essential… 
[PDF] Semantic Reader
13 Citations
Background Citations
7
Methods Citations
6
Results Citations
2

13 Citations

Action Unit Detection by Learning the Deformation Coefficients of a 3D Morphable Model

The proposed approach for AU detection is validated on the Bosphorus dataset, reporting competitive results with respect to the state-of-the-art, even in a challenging cross-dataset setting.

A 3DMM-Based Framework for Deformation Measurement in Face Rehabilitation

Results indicated that the proposed approach can accurately capture the face deformation even for localized and asymmetric ones, and these results were obtained just using RGB targets without the need for 3D scans captured with costly devices.

Generating Complex 4D Expression Transitions by Learning Face Landmark Trajectories

A new model that generates transitions between different expressions, and synthesizes long and composed 4D expressions is proposed, and brings improvements with respect to previous solutions, while retaining good generalization to unseen data.

Sparse to Dense Dynamic 3D Facial Expression Generation

The proposed solution to the task of generating dynamic 3D facial expressions from a neutral 3D face and an expression label brings significant improvements with respect to previous solutions in terms of both dynamic expression generation and mesh reconstruction, while retaining good generalization to unseen data.

3D-aware Blending with Generative NeRFs

3D-aware blending that directly blends images on the NeRF's latent representation space, rather than raw pixel space is proposed, which outperforms existing 2D baselines, as validated by extensive quantitative and qualitative evaluations with FFHQ and AFHQ-Cat.

The Florence 4D Facial Expression Dataset

This paper proposes a large dataset, named Florence 4D, composed of dynamic sequences of 3D face models, where a combination of synthetic and real identities exhibit an unprecedented variety of 4D facial expressions, with variations that include the classical neutral-apex transition, but generalize to expression-to-expression.

Learning Streamed Attention Network from Descriptor Images for Cross-Resolution 3D Face Recognition

A hybrid framework for cross-resolution 3D face recognition which utilizes a Streamed Attention Network (SAN) that combines handcrafted features with Convolutional Neural Networks (CNNs) that allows a better exploitation of the richer information provided by low-level features, resulting in improved recognition.

Reconstructing 3D Face of Infants in Social Interactions Using Morphable Models of Non-Infants.

  • E. SariyanidiC. Zampella B. Tunç
  • Psychology
    Eurographics ... Workshop on 3D Object Retrieval : EG 3DOR. Eurographics Workshop on 3D Object Retrieval
  • 2022
Five state-of-the-art 3DMM fitting methods are compared on data from naturalistic infant-caregiver interactions to suggest that it is possible to produce consistent and subject-specific reconstructions of 3D shape identity from multiple frames, but not from a single frame.

Measuring 3D face deformations from RGB images of expression rehabilitation exercises

FaceTuneGAN: Face Autoencoder for Convolutional Expression Transfer Using Neural Generative Adversarial Networks

References

SHOWING 1-10 OF 87 REFERENCES

A Dictionary Learning-Based 3D Morphable Shape Model

A new approach for constructing a 3D morphable shape model (called DL-3DMM) is proposed and it is shown the solution can reach the accuracy of deformation required in applications where fine details of the face are concerned.

Deep, dense and accurate 3D face correspondence for generating population specific deformable models

3D Face Modeling From Diverse Raw Scan Data

An innovative framework to jointly learn a nonlinear face model from a diverse set of raw 3D scan databases and establish dense point-to-point correspondence among their scans is proposed and its contribution to single-image 3D face reconstruction is demonstrated.

Boosting Local Shape Matching for Dense 3D Face Correspondence

The formulated deformation satisfies coherent local motions everywhere on a face, and the experimental results demonstrate that the proposed method can achieve accurate feature correspondence, coherent local shape motion, and compact data representation.

Dense Semantic and Topological Correspondence of 3D Faces without Landmarks

A general framework for dense correspondence of 3D faces without landmarks is proposed in this paper, and the dense correspondence goal is revisited in two perspectives: semantic and topological correspondence.

Dictionary Learning Based 3D Morphable Model Construction for Face Recognition with Varying Expression and Pose

A new approach for constructing a 3D morph able model (3DMM) is proposed, constructed by learning a dictionary of basis components, instead of using the traditional approach based on PCA decomposition and its application to face recognition is experimented.

A 3D Morphable Model of Craniofacial Shape and Texture Variation

We present a fully automatic pipeline to train 3D Morphable Models (3DMMs), with contributions in pose normalisation, dense correspondence using both shape and texture information, and high quality,…

Establishing Point Correspondence of 3D Faces Via Sparse Facial Deformable Model

A sparse facial deformable model to automatically achieve a dense vertex-to-vertex anthropometric correspondence between 3D faces is proposed, and a patch-based sparse representation is proposed as the correspondence constraint.

A 3D Morphable Model Learnt from 10,000 Faces

The presented extensive qualitative and quantitative evaluations reveal that the proposed 3DMM achieves state-of-the-art results, outperforming existing models by a large margin.

Dense 3D Face Correspondence

An algorithm that automatically establishes dense correspondences between a large number of 3D faces and iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model.
...