Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision

@article{Sanyal2019LearningTR,
  title={Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision},
  author={Soubhik Sanyal and Timo Bolkart and Haiwen Feng and Michael J. Black},
  journal={2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2019},
  pages={7755-7764}
}
The estimation of 3D face shape from a single image must be robust to variations in lighting, head pose, expression, facial hair, makeup, and occlusions. Robustness requires a large training set of in-the-wild images, which by construction, lack ground truth 3D shape. To train a network without any 2D-to-3D supervision, we present RingNet, which learns to compute 3D face shape from a single image. Our key observation is that an individual’s face shape is constant across images, regardless of… 

3D Face From X: Learning Face Shape From Diverse Sources

TLDR
Experimental results demonstrate that with training data from more sources, the proposed method to learn a unified face model can learn a more powerful face model.

End-to-End 3D Face Reconstruction with Expressions and Specular Albedos from Single In-the-wild Images

TLDR
This paper proposes a convolutional neural network based framework to regress the face model from a single image in the wild, and uses novel hybrid loss functions to disentangle face shape identi-ties, expressions, poses, albedos, and lighting.

Learning 3D Face Reconstruction with a Pose Guidance Network

TLDR
A self-supervised learning approach to learning monocular 3D face reconstruction with a pose guidance network (PGN) that combines the complementary strengths of parametric model learning and data-driven learning techniques.

Self-Supervised 3D Face Reconstruction via Conditional Estimation

TLDR
A conditional estimation framework to learn 3D facial parameters from 2D single-view images by self-supervised training from videos that can be efficiently trained with in-the-wild video clips and incorporates the reflectance symmetry and consistency.

Learning Complete 3D Morphable Face Models from Images and Videos

TLDR
This work presents the first approach to learn complete 3D models of face identity and expression geometry, and reflectance, just from images and videos, and shows that the learned models better generalize and lead to higher quality image-based reconstructions than existing approaches.

Learning an animatable detailed 3D face model from in-the-wild images

TLDR
This work presents the first approach that regresses 3D face shape and animatable details that are specific to an individual but change with expression, and introduces a novel detail-consistency loss that disentangles person-specific details from expression-dependent wrinkles.

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction

TLDR
A novel algorithm is proposed to predict elaborate riggable 3D face models from a single image input using a deep neural network to evaluate single-view facial 3D reconstruction.

Deep Facial Non-Rigid Multi-View Stereo

TLDR
This method optimizes the 3D face shape by explicitly enforcing multi-view appearance consistency, which is known to be effective in recovering shape details according to conventional multi- view stereo methods.

Learning to regulate 3D head shape by removing occluding hair from in-the-wild images

TLDR
The unsupervised 3DMM model achieves state-of-the-art results on popular benchmarks and can be used to infer the head shape, facial features, and textures for direct use in animation or avatar creation.

Learning 3D Faces from Photo-Realistic Facial Synthesis

TLDR
This work uses a deep neural network to regress vertex coordinates from the given image and then refine them by a non-rigid deformation process to more accurately capture local shape similarity.
...

References

SHOWING 1-10 OF 40 REFERENCES

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

TLDR
This work proposes to address many of these limitations by training a Convolutional Neural Network (CNN) on an appropriate dataset consisting of 2D images and 3D facial models or scans, and achieves this via a simple CNN architecture that performs direct regression of a volumetric representation of the3D facial geometry from a single 2D image.

Expressive Body Capture: 3D Hands, Face, and Body From a Single Image

TLDR
This work uses the new method, SMPLify-X, to fit SMPL-X to both controlled images and images in the wild, and evaluates 3D accuracy on a new curated dataset comprising 100 images with pseudo ground-truth.

Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network

TLDR
A robust method for regressing discriminative 3D morphable face models (3DMM) using a convolutional neural network to regress 3DMM shape and texture parameters directly from an input photo is described.

Face reconstruction in the wild

TLDR
This work addresses the problem of reconstructing 3D face models from large unstructured photo collections, e.g., obtained by Google image search or from personal photo collections in iPhoto, and leverages multi-image shading, but unlike traditional photometric stereo approaches, allows for changes in viewpoint and shape.

Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz

TLDR
This first approach that jointly learns a regressor for face shape, expression, reflectance and illumination on the basis of a concurrently learned parametric face model is presented, which compares favorably to the state-of-the-art in terms of reconstruction quality, better generalizes to real world faces, and runs at over 250 Hz.

Nonlinear 3D Face Morphable Model

  • Luan TranXiaoming Liu
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
TLDR
This paper proposes an innovative framework to learn a nonlinear 3DMM model from a large set of unconstrained face images, without collecting 3D face scans, and demonstrates the superior representation power of the nonlinear 2D Morphable Model over its linear counterpart.

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

TLDR
A straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment and surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.

Face Alignment Across Large Poses: A 3D Solution

TLDR
3D Dense Face Alignment (3DDFA), in which a dense 3D face model is fitted to the image via convolutional neutral network (CNN), is proposed, and a method to synthesize large-scale training samples in profile views to solve the third problem of data labelling is proposed.

Generating 3D faces using Convolutional Mesh Autoencoders

TLDR
This work introduces a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface and shows that, replacing the expression space of an existing state-of-the-art face model with this model, achieves a lower reconstruction error.

Reconstruction of Personalized 3D Face Rigs from Monocular Video

TLDR
A novel approach for the automatic creation of a personalized high-quality 3D face rig of an actor from just monocular video data, based on three distinct layers that allow the actor's facial shape as well as capture his person-specific expression characteristics at high fidelity, ranging from coarse-scale geometry to fine-scale static and transient detail on the scale of folds and wrinkles.