Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

@article{Sela2017UnrestrictedFG,
  title={Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation},
  author={Matan Sela and Elad Richardson and Ron Kimmel},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
  year={2017},
  pages={1585-1594}
}
It has been recently shown that neural networks can recover the geometric structure of a face from a single given image. A common denominator of most existing face geometry reconstruction methods is the restriction of the solution space to some low-dimensional subspace. While such a model significantly simplifies the reconstruction problem, it is inherently limited in its expressiveness. As an alternative, we propose an Image-to-Image translation network that jointly maps the input image to a… 
Learning 3D Faces from Photo-Realistic Facial Synthesis
TLDR
This work uses a deep neural network to regress vertex coordinates from the given image and then refine them by a non-rigid deformation process to more accurately capture local shape similarity.
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
TLDR
A straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment and surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.
SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery
TLDR
A novel photometric optimization method that recovers detailed facial geometry from a single image in an unsupervised manner that achieves state-of-the-art on facial geometric detail recovery, using only a single in the-wild image.
High-fidelity facial reflectance and geometry inference from an unconstrained image
TLDR
A deep learning-based technique to infer high-quality facial reflectance and geometry given a single unconstrained image of the subject, which may contain partial occlusions and arbitrary illumination conditions, and demonstrates the rendering of high-fidelity 3D avatars from a variety of subjects captured under different lighting conditions.
End-to-End 3D Facial Shape Reconstruction From an Unconstrained Image
TLDR
A novel architecture is proposed to learn face models in nonlinear spaces using a Convolutional Neural Networks based End-to-End 3D Facial Shape Reconstruction from a single-view 2D image scheme to improve the results of reconstructed face model.
Learning Monocular Face Reconstruction using Multi-View Supervision
TLDR
A method to reconstruct faces from a single portrait image by combining supervised losses on synthetic data with indirect supervision on real data using a novel multi-view photo-consistency loss and regularize the depth estimation using a 3D morphable model (3DMM).
use iterative error feedback [ 12 ] to optimize the shape parameters of a grayscale morphable model from a single input image
TLDR
This work introduces InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image and demonstrates that this strategy outperforms completely synthetically trained networks.
Self-Supervised Learning of Detailed 3D Face Reconstruction
TLDR
An end-to-end learning framework for detailed 3D face reconstruction from a single image that combines a photometric loss and a facial perceptual loss between the input face and the rendered face, and a displacement map in UV-space to represent a3D face.
Lightweight Photometric Stereo for Facial Details Recovery
TLDR
This paper proposes a novel neural network specially designed for photometric stereo based 3D face reconstruction that only requires sparse inputs or even a single image to recover high-fidelity face shapes with images captured under near-field lights.
InverseFaceNet: Deep Monocular Inverse Face Rendering
TLDR
This work introduces InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image and demonstrates that this strategy outperforms completely synthetically trained networks.
...
...

References

SHOWING 1-10 OF 55 REFERENCES
Learning Detailed Face Reconstruction from a Single Image
TLDR
This work proposes to leverage the power of convolutional neural networks to produce a highly detailed face reconstruction from a single image, and introduces an end- to-end CNN framework which derives the shape in a coarse-to-fine fashion.
3D Face Reconstruction by Learning from Synthetic Data
TLDR
The proposed approach is based on a Convolutional-Neural-Network (CNN) which extracts the face geometry directly from its image and successfully recovers facial shapes from real images, even for faces with extreme expressions and under various lighting conditions.
3D Face Reconstruction With Geometry Details From a Single Image
TLDR
This work develops a novel method for reconstructing 3D faces from unconstrained 2D images using a coarse-to-fine optimization strategy that outperforms the state-of-the-art approaches in terms of accuracy and detail recovery.
3D Face Reconstruction from a Single Image Using a Single Reference Face Shape
TLDR
This work proposes a novel method for 3D shape recovery of faces that exploits the similarity of faces, and obtains as input a single image and uses a mere single 3D reference model of a different person's face.
Photorealistic Facial Texture Inference Using Deep Neural Networks
TLDR
A data-driven inference method is presented that can synthesize a photorealistic texture map of a complete 3D face model given a partial 2D view of a person in the wild and successful face reconstructions from a wide range of low resolution input images are demonstrated.
Cascaded Regressor based 3D Face Reconstruction from a Single Arbitrary View Image
TLDR
This paper approaches the 3D face reconstruction problem as a regression problem rather than a model fitting problem, and can reconstruct 3D faces from arbitrary view images more efficiently and more accurately than existing methods.
Robust 3D Face Shape Reconstruction from Single Images via Two-Fold Coupled Structure Learning and Off-the-Shelf Landmark Detectors
TLDR
This paper proposes a robust method for monocular face shape reconstruction (MFSR) using a sparse set of facial landmarks that are detected by most of the off-theshelf landmark detectors, and forms the MFSR problem as a Two-Fold Coupled Structure Learning (2FCSL) process, which consists of learning a regression between two subspaces spanned by 3D sparse landmarks and 2D sparse landmark.
Reconstruction of Personalized 3D Face Rigs from Monocular Video
TLDR
A novel approach for the automatic creation of a personalized high-quality 3D face rig of an actor from just monocular video data, based on three distinct layers that allow the actor's facial shape as well as capture his person-specific expression characteristics at high fidelity, ranging from coarse-scale geometry to fine-scale static and transient detail on the scale of folds and wrinkles.
Effective face frontalization in unconstrained images
TLDR
This work explores the simpler approach of using a single, unmodified, 3D surface as an approximation to the shape of all input faces, and shows that this leads to a straightforward, efficient and easy to implement method for frontalization.
High-fidelity Pose and Expression Normalization for face recognition in the wild
TLDR
A High-fidelity Pose and Expression Normalization (HPEN) method with 3D Morphable Model (3DMM) which can automatically generate a natural face image in frontal pose and neutral expression and an inpainting method based on Possion Editing to fill the invisible region caused by self occlusion is proposed.
...
...