Face/Off: live facial puppetry

@inproceedings{Weise2009FaceOffLF,
  title={Face/Off: live facial puppetry},
  author={Thibaut Weise and Hao Li and Luc Van Gool and Mark Pauly},
  booktitle={SCA '09},
  year={2009}
}
We present a complete integrated system for live facial puppetry that enables high-resolution real-time facial expression tracking with transfer to another person's face. The system utilizes a real-time structured light scanner that provides dense 3D data and texture. A generic template mesh, fitted to a rigid reconstruction of the actor's face, is tracked offline in a training stage through a set of expression sequences. These sequences are used to build a person-specific linear face model… 

Real-time facial animation from live video tracking

TLDR
A complete pipe-line of a practical system for producing real-time facial expressions of a 3D virtual avatar controlled by an actor's live performances and handles practical challenges arising from markerless expression captures from a single conventional video camera is described.

3D shape regression for real-time facial animation

TLDR
A real-time performance-driven facial animation system based on 3D shape regression that learns an accurate, user-specific face alignment model from an easily acquired set of training data, generated from images of the user performing a sequence of predefined facial poses and expressions.

Face It!: A Pipeline for Real-Time Performance-Driven Facial Animation

TLDR
This paper transfers facial expressions from 2D images to a 3D virtual character, by estimating the rigid head pose and non-rigid face deformation from detected and tracked 2D facial landmarks, and forms a lightweight energy-based optimization problem.

Performance-Driven Facial Expression Real-Time Animation Generation

TLDR
Experimental results show that the proposed novel method of real-time facial expression reconstruction can reconstruct facial expression efficiency in low-cost.

Performance-Driven Facial Expression Real-Time Animation Generation

TLDR
Experimental results show that the proposed novel method of real-time facial expression reconstruction can reconstruct facial expression efficiency in low-cost.

Real-time facial animation with image-based dynamic avatars

We present a novel image-based representation for dynamic 3D avatars, which allows effective handling of various hairstyles and headwear, and can generate expressive facial animations with fine-scale

Real-time avatar animation from a single image

TLDR
A real time facial puppetry system that requires no special hardware, runs in real time (23 frames-per-second), and requires only a single image of the avatar and user is presented.

Real-time avatar animation from a single image

TLDR
A real time facial puppetry system that requires no special hardware, runs in real time (23 frames-per-second), and requires only a single image of the avatar and user is presented.

Facial Animation Based on 2D Shape Regression

TLDR
Experiments show that the facial animation system can accurately detect facial landmarks and the animation results are visually plausible and similar to the user’s facial motion.

Automatic Facial Expression Tracking for 4D Range Scans

TLDR
A Scaling Iterative Closest Points (SICP) algorithm is introduced to compute the optimal rigid registration between a template facial model and a range scan with consideration of the scale problem.
...

References

SHOWING 1-10 OF 49 REFERENCES

Resynthesizing facial animation through 3D model-based tracking

Given video footage of a person's face, we present new techniques to automatically recover the face position and the facial expression from each frame in the video sequence. A 3D face model is fitted

Physically-based facial modelling, analysis, and animation

TLDR
A new 3D hierarchical model of the human face is developed that incorporates a physically-based approximation to facial tissue and a set of anatomically-motivated facial muscle actuators and is efficient enough to produce facial animation at interactive rates on a high-end graphics workstation.

Vision-based control of 3D facial animation

TLDR
A real-time facial tracking system to extract a small set of animation control parameters from video and an efficient expression retargeting technique whose run-time computation is constant independent of the complexity of the character model.

Pose-space animation and transfer of facial details

This paper presents a novel method for real-time animation of highly-detailed facial expressions based on a multi-scale decomposition of facial geometry into large-scale motion and fine-scale

Geometry-driven photorealistic facial expression synthesis

TLDR
A geometry-driven facial expression synthesis system that automatically synthesizes a corresponding expression image that includes photorealistic and natural looking expression details and a technique to infer the missing feature point motions from the tracked subset by using an example-based approach.

Animating blendshape faces by cross-mapping motion capture data

TLDR
This work presents a semi-automatic technique to directly animate popularized 3D blendshape face models by mapping facial motion capture data spaces to 3D blendsshape face spaces by using the Radial Basis Function (RBF) regression technique.

Human face project

TLDR
The face tracking system developed is exemplary of model-based computer vision, and exploits the detailed degrees of freedom of a geometric face model to confine the space of solutions.

Face poser: interactive modeling of 3D facial expressions using model priors

TLDR
An intuitive interface for interactively posing 3D facial expressions by combining the prior with user-defined constraints and Maximizing the posterior allows us to generate an optimal and natural facial expression that satisfies the user- defined constraints.

Performance-driven facial animation

TLDR
A means of acquiring the expressions of real faces, and applying them to computer-generated faces is described as an "electronic mask" that offers a means for the traditional talents of actors to be flexibly incorporated in digital animations.

Performance-driven facial animation

TLDR
A means of acquiring the expressions of real faces, and applying them to computer-generated faces is described as an "electronic mask" that offers a means for the traditional talents of actors to be flexibly incorporated in digital animations.