3D Morphable Face Models—Past, Present, and Future

@article{Egger20203DMF,
  title={3D Morphable Face Models—Past, Present, and Future},
  author={Bernhard Egger and W. Smith and Ayush Tewari and Stefanie Wuhrer and Michael Zollh{\"o}fer and Thabo Beeler and Florian Bernard and Timo Bolkart and Adam Kortylewski and Sami Romdhani and Christian Theobalt and Volker Blanz and Thomas Vetter},
  journal={ACM Transactions on Graphics (TOG)},
  year={2020},
  volume={39},
  pages={1 - 38}
}
In this article, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed. The challenges in building and applying these models, namely, capture, modeling, image formation, and image analysis, are still active research topics, and we review the state-of-the-art in each of these areas. We also look ahead, identifying unsolved challenges, proposing directions for future research, and highlighting the broad range of current and future applications. 

Figures and Tables from this paper

A Morphable Face Albedo Model

The model is demonstrated in a state of the art analysis-by-synthesis 3DMM fitting pipeline, are the first to integrate specular map estimation and outperform the Basel Face Model in albedo reconstruction.

Survey on 3D face reconstruction from uncalibrated images

Learning Complete 3D Morphable Face Models from Images and Videos

This work presents the first approach to learn complete 3D models of face identity and expression geometry, and reflectance, just from images and videos, and shows that the learned models better generalize and lead to higher quality image-based reconstructions than existing approaches.

ReDA:Reinforced Differentiable Attribute for 3D Face Reconstruction

This work introduces a new free-form deformation layer that sits on top of 3DMM to enjoy both the prior knowledge and out-of-space modeling, and makes the rendering to be more differentiable through a set of convolution operations with multi-scale kernel sizes.

3D Face Reconstruction in Deep Learning Era: A Survey

  • Sahil SharmaVijay Kumar
  • Computer Science
    Archives of computational methods in engineering : state of the art reviews
  • 2022
An in-depth analysis of 3D face reconstruction using deep learning techniques and the performance analysis of different face reconstruction techniques in terms of software, hardware, pros and cons is provided.

Research on Current Situation of 3D face reconstruction based on 3D Morphable Models

The main technical methods and research status of 3D face reconstruction based on 3D morphable models in recent 20 years are introduced, the advantages and disadvantages and applicability of various researches on 3d face reconstruction using 3dmm are summarized, and the current research hotspot and future development trend are analyzed.

End-to-End 3D Face Reconstruction with Expressions and Specular Albedos from Single In-the-wild Images

This paper proposes a convolutional neural network based framework to regress the face model from a single image in the wild, and uses novel hybrid loss functions to disentangle face shape identities, expressions, poses, albedos, and lighting.

i3DMM: Deep Implicit 3D Morphable Model of Human Heads

The first deep implicit 3D morphable model (i3DMM) of full heads, which not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair is presented.

Towards a Complete 3D Morphable Model of the Human Head

The most complete 3DMM of the human head to date is presented that includes face, cranium, ears, eyes, teeth and tongue and is used to reconstruct full head representations from single, unconstrained images allowing us to parameterize craniofacial shape and texture.

Implicit Neural Deformation for Multi-View Face Reconstruction

To handle in-the-wild sparse-view input of the same target with different expressions at test time, a novel view-switch loss to enforce consistency among different views is proposed and residual latent code is proposed to effectively expand the shape space of the learned implicit face representation.
...

References

SHOWING 1-10 OF 406 REFERENCES

Optimal Regions for Linear Model-Based 3D Face Reconstruction

This paper proposes algorithms that are capable of automatically finding the optimal subdivision given a training set and the number of desired regions in region-based 3D representations of the human face.

2D face pose normalisation using a 3D morphable model

The results on the non frontal XM2VTS database showed that pose correction can improve recognition rates up to 30% and state of the art 2D recognition algorithm is used.

A Morphable Face Albedo Model

The model is demonstrated in a state of the art analysis-by-synthesis 3DMM fitting pipeline, are the first to integrate specular map estimation and outperform the Basel Face Model in albedo reconstruction.

Face identification across different poses and illuminations with a 3D morphable model

  • V. BlanzS. RomdhaniT. Vetter
  • Computer Science
    Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition
  • 2002
We present a novel approach for recognizing faces in images taken from different directions and under different illumination. The method is based on a 3D morphable face model that encodes shape and

Adaptive 3D Face Reconstruction from Unconstrained Photo Collections

This paper fits a 3D Morphable Model to form a personalized template and develops a novel photometric stereo formulation, under a coarse-to-fine scheme, to adapt to low quality photo collections with fewer images.

Combining 3D Morphable Models: A Large Scale Face-And-Head Model

This work proposes two methods for solving the problem of combining two or more 3DMMs that are built using different templates that perhaps only partly overlap, have different representation capabilities and are built from different datasets that may not be publicly-available.

Inverse Rendering of Faces with a 3D Morphable Model

  • O. Aldrian
  • Computer Science, Mathematics
    IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2013
This paper starts by recovering 3D shape using a novel algorithm which incorporates generalization error of the model obtained from empirical measurements, and describes two methods to recover facial texture, diffuse lighting, specular reflectance, and camera properties from a single image.

3D Face Morphable Models "In-the-Wild"

This paper proposes the first, to the best of the knowledge, in-the-wild 3DMM by combining a powerful statistical model of facial shape, which describes both identity and expression, with an in- the-wild texture model, and demonstrates the first 3D facial database with relatively unconstrained conditions.

3D morphable face models revisited

  • Ankur PatelW. Smith
  • Computer Science
    2009 IEEE Conference on Computer Vision and Pattern Recognition
  • 2009
It is demonstrated how the statistical tools of thin-plate splines and Procrustes analysis can be used to construct a morphable model that is both more efficient and generalises to novel face surfaces more accurately than previous models.

A Multiresolution 3D Morphable Face Model and Fitting Framework

The Surrey Face Model is presented, a multi-resolution 3D Morphable Model that is made available to the public for non-commercial purposes and a lightweight open-source C++ library designed with simplicity and ease of integration as its foremost goals.
...