HeadFusion: 360° Head Pose Tracking Combining 3D Morphable Model and 3D Reconstruction

@article{Yu2018HeadFusion3H,
  title={HeadFusion: 360° Head Pose Tracking Combining 3D Morphable Model and 3D Reconstruction},
  author={Yuechen Yu and Kenneth Alberto Funes Mora and Jean-Marc Odobez},
  journal={IEEE transactions on pattern analysis and machine intelligence},
  year={2018},
  volume={40 11},
  pages={
          2653-2667
        }
}
Head pose estimation is a fundamental task for face and social related research. Although 3D morphable model (3DMM) based methods relying on depth information usually achieve accurate results, they usually require frontal or mid-profile poses which preclude a large set of applications where such conditions can not be garanteed, like monitoring natural interactions from fixed sensors placed in the environment. A major reason is that 3DMM models usually only cover the face region. In this paper… 
Visibility Constrained Generative Model for Depth-Based 3D Facial Pose Tracking
TLDR
A statistical 3D morphable model that flexibly describes the distribution of points on the surface of the face model, with an efficient switchable online adaptation that gradually captures the identity of the tracked subject and rapidly constructs a suitable face model when the subject changes is introduced.
Head Pose Estimation through Keypoints Matching between Reconstructed 3D Face Model and 2D Image
TLDR
A method which does not need to be trained with head pose labels, but matches the keypoints between a reconstructed 3D face model and the 2D input image, for head pose estimation that achieves excellent cross-dataset performance and surpasses most of the existing state-of-the-art approaches.
Temporal Head Pose Estimation From Point Cloud in Naturalistic Driving Conditions
TLDR
This work proposes a novel temporal deep learning model for head pose estimation from point cloud, leveraging the 3D spatial structure of the face and shows quantitatively and qualitatively that incorporating temporal information provides large improvements not only in accuracy, but also in the smoothness of the predictions.
DD-Pose - A large-scale Driver Head Pose Benchmark
We introduce DD-Pose, the Daimler TU Delft Driver Head Pose Benchmark, a large-scale and diverse benchmark for image-based head pose estimation and driver analysis. It contains 330k measurements from
Robust head pose estimation based on key frames for human-machine interaction
TLDR
A head pose estimation framework that combines 2D and 3D cues using the concept of key frames (KFs) that can handle partial occlusions and extreme rotations even with noisy depth data, improving the accuracy of pose estimation and detection rate.
Efficient 3D Face Recognition in Uncontrolled Environment
TLDR
An efficient pose fusion algorithm is developed that frontalizes the faces and combines the multiple inputs, and a new 3D registration method based on the unified coordinate system (UCS) is introduced to compensate for pose and scale variations and normalize the probe and gallery face.
FASHE: A FrActal Based Strategy for Head Pose Estimation
TLDR
FASHE, an approach based on partitioned iterated function systems (PIFS) to represent auto-similarities within face image through a contractive affine function transforming the domain blocks extracted only once by a single frontal reference image, in a good approximation of the range blocks which the target image has been partitioned into is presented.
A survey of head pose estimation methods
  • X. Shao, Z. Qiang, Hong Lin, Yueyu Dong, Xiaorui Wang
  • Computer Science
    2020 International Conferences on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics (Cybermatics)
  • 2020
TLDR
A comparison of computer systems head pose estimating methods focuses on their capabilities, estimates of rough and fine head pose, and the prominent method is very suitable for unconstrained environments.
Deep Convolutional Neural Network-based Bernoulli Heatmap for Head Pose Estimation
...
...

References

SHOWING 1-10 OF 54 REFERENCES
Robust and Accurate 3D Head Pose Estimation through 3DMM and Online Head Model Reconstruction
TLDR
A robust head pose estimation framework is presented by complementing a 3DMM model with an online 3D reconstruction of the full head providing more support when handling extreme head poses, and achieves state-of-the-art pose estimation accuracy on the BIWI dataset.
Augmented Blendshapes for Real-Time Simultaneous 3D Head Modeling and Facial Motion Capture
TLDR
This framework is the first one to provide simultaneously comprehensive facial motion tracking and a detailed 3D model of the user's head and demonstrates robust and high-fidelity simultaneous facialMotion tracking and 3D head modeling results on a wide range of subjects with various head poses and facial expressions.
Gaze Estimation in the 3D Space Using RGB-D Sensors
TLDR
This work proposes to leverage the depth data of RGB-D cameras to perform an accurate head pose tracking, acquire head pose invariance through a 3D rectification process that renders head pose dependent eye images into a canonical viewpoint, and computes the line-of-sight in the 3D space.
A data-driven model for monocular face tracking
TLDR
It is demonstrated that a data-driven approach for model construction is suitable for tracking non rigid objects and offers an elegant and practical alternative to the task of manual construction of models using 3D scanners or CAD modelers.
Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting
TLDR
This paper proposes a face alignment method for large-pose face images, by combining the powerful cascaded CNN regressor method and 3DMM, and forms the face alignment as a3DMM fitting problem, where the camera projection matrix and3D shape parameters are estimated by a cascade of CNN-based regressors.
Gaze estimation from multimodal Kinect data
TLDR
A multimodal method that rely on depth sensing to obtain robust and accurate head pose tracking even under large head pose, and on the visual data to obtain the remaining eye-in-head gaze directional information from the eye image is proposed.
Real-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion
TLDR
A new system for real-time dense reconstruction with equivalent quality to existing online methods, but with support for additional spatial scale and robustness in dynamic scenes, designed around a simple and flat point-Based representation.
FaceWarehouse: A 3D Facial Expression Database for Visual Computing
TLDR
There is a much richer matching collection of expressions, enabling depiction of most human facial actions, in FaceWarehouse, a database of 3D facial expressions for visual computing applications.
A morphable model for the synthesis of 3D faces
TLDR
A new technique for modeling textured 3D faces by transforming the shape and texture of the examples into a vector space representation, which regulates the naturalness of modeled faces avoiding faces with an “unlikely” appearance.
It’s Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation
TLDR
This work proposes an appearance-based method that, in contrast to a long-standing line of work in computer vision, only takes the full face image as input, and encodes the face image using a convolutional neural network with spatial weights applied on the feature maps to flexibly suppress or enhance information in different facial regions.
...
...