Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

@article{Jackson2017LargeP3,
  title={Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression},
  author={Aaron S. Jackson and Adrian Bulat and Vasileios Argyriou and Georgios Tzimiropoulos},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
  year={2017},
  pages={1031-1039}
}
3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. [] Key Method We achieve this via a simple CNN architecture that performs direct regression of a volumetric representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related task of facial landmark localization can be incorporated into the proposed framework and help improve reconstruction quality, especially for the cases of large poses and facial expressions. Code and models will…
View on IEEE
View With Semantic Reader
329 Citations
Highly Influential Citations
54
Background Citations
129
Methods Citations
153
Results Citations
15

329 Citations

Citation Type

Citation Type

End-to-End 3D Facial Shape Reconstruction From an Unconstrained Image
TLDR
A novel architecture is proposed to learn face models in nonlinear spaces using a Convolutional Neural Networks based End-to-End 3D Facial Shape Reconstruction from a single-view 2D image scheme to improve the results of reconstructed face model.
Joint Face Alignment and 3D Face Reconstruction with Efficient Convolution Neural Networks
TLDR
An efficient and lightweight network model combined with Depthwise Separable Convolution and Muti-scale Representation, Lightweight Attention Mechanism, named Mobile-FRNet is designed, which significantly improves the accuracy of face alignment and 3D face reconstruction.
3D Face Reconstruction from Light Field Images: A Model-free Approach
TLDR
This method is model-free, requires only a few training samples to learn FaceLFnet and can reconstruct 3D faces with high accuracy from single light field images under varying poses, expressions and lighting conditions.
3D Single Image Face Reconstruction Approaches With Deep Neural Networks
TLDR
3D facial reconstruction algorithms used for application such as 3D printing, 3D VR games and facial recognition, and 3D face learning methods developed in 2019 are discussed.
3D Face Reconstruction from Low-Resolution Images with Convolutional Neural Networks
TLDR
This work introduces an end-to-end trainable CNN capable of simultaneously estimating 3D geometry and pose of a face given a single LR image, and coupling this network with a state-of-the-art LR face detector, builds a 3D face reconstruction pipeline ready for integration into real-world applications.
Robust 3D Face Reconstruction Using One/Two Facial Images
TLDR
A novel CNN-based method which targets 3D facial reconstruction from two facial images, one in front and one from the side, as are often available to law enforcement agencies (LEAs), trained on both synthetic and real facial data.
Lightweight Photometric Stereo for Facial Details Recovery
TLDR
This paper proposes a novel neural network specially designed for photometric stereo based 3D face reconstruction that only requires sparse inputs or even a single image to recover high-fidelity face shapes with images captured under near-field lights.
A Fast and Robust, Forehead-Augmented 3D Face Reconstruction from Multiple Images using Geometrical Methods
TLDR
This work proposes a mostly geometric method for facial reconstruction, based on structure from motion techniques on uncalibrated cameras, augmented with forehead surface modelling for added realism, and proves that it is on par with state of the art deep learning techniques.
Detailed 3D Face Reconstruction from a Single RGB Image
TLDR
This paper introduces a method to obtain a detailed 3D reconstruction of facial skin from a single RGB image without requiring any information about the observed material nor training data to model the wrinkle properties, and outperforms the current approaches regarding3D reconstruction accuracy.
D Face Reconstruction from Light Field Images : A Model-free Approach ( This a preprint of the paper accepted in ECCV 2018 )
TLDR
This method is model-free, requires only a few training samples to learn FaceLFnet and can reconstruct 3D faces with high accuracy from single light field images under varying poses, expressions and lighting conditions.
...
...

References

SHOWING 1-10 OF 32 REFERENCES
Learning Detailed Face Reconstruction from a Single Image
TLDR
This work proposes to leverage the power of convolutional neural networks to produce a highly detailed face reconstruction from a single image, and introduces an end- to-end CNN framework which derives the shape in a coarse-to-fine fashion.
Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting
TLDR
This paper proposes a face alignment method for large-pose face images, by combining the powerful cascaded CNN regressor method and 3DMM, and forms the face alignment as a3DMM fitting problem, where the camera projection matrix and3D shape parameters are estimated by a cascade of CNN-based regressors.
Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network
TLDR
A robust method for regressing discriminative 3D morphable face models (3DMM) using a convolutional neural network to regress 3DMM shape and texture parameters directly from an input photo is described.
Face Alignment Across Large Poses: A 3D Solution
TLDR
3D Dense Face Alignment (3DDFA), in which a dense 3D face model is fitted to the image via convolutional neutral network (CNN), is proposed, and a method to synthesize large-scale training samples in profile views to solve the third problem of data labelling is proposed.
Face reconstruction in the wild
TLDR
This work addresses the problem of reconstructing 3D face models from large unstructured photo collections, e.g., obtained by Google image search or from personal photo collections in iPhoto, and leverages multi-image shading, but unlike traditional photometric stereo approaches, allows for changes in viewpoint and shape.
3D Face Reconstruction from a Single Image Using a Single Reference Face Shape
TLDR
This work proposes a novel method for 3D shape recovery of faces that exploits the similarity of faces, and obtains as input a single image and uses a mere single 3D reference model of a different person's face.
Joint Face Alignment and 3D Face Reconstruction
TLDR
The proposed method iteratively and alternately applies two sets of cascaded regressors, one for updating 2D landmarks and the other for updating reconstructed pose-expression-normalized (PEN) 3D face shape, to simultaneously solve the two problems of face alignment and3D face reconstruction from an input 2D face image of arbitrary poses and expressions.
A 3D Face Model for Pose and Illumination Invariant Face Recognition
TLDR
This paper publishes a generative 3D shape and texture model, the Basel Face Model (BFM), and demonstrates its application to several face recognition task and publishes a set of detailed recognition and reconstruction results on standard databases to allow complete algorithm comparisons.
A Multiresolution 3D Morphable Face Model and Fitting Framework
TLDR
The Surrey Face Model is presented, a multi-resolution 3D Morphable Model that is made available to the public for non-commercial purposes and a lightweight open-source C++ library designed with simplicity and ease of integration as its foremost goals.
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
TLDR
This paper proposes a fine discretization of the 3D space around the subject and trains a ConvNet to predict per voxel likelihoods for each joint, which creates a natural representation for 3D pose and greatly improves performance over the direct regression of joint coordinates.
...
...