OpenFace: An open source facial behavior analysis toolkit

@article{Baltruaitis2016OpenFaceAO,
  title={OpenFace: An open source facial behavior analysis toolkit},
  author={Tadas Baltru{\vs}aitis and Peter Robinson and Louis-Philippe Morency},
  journal={2016 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  year={2016},
  pages={1-10}
}
Over the past few years, there has been an increased interest in automatic facial behavior analysis and understanding. [] Key Result Finally, OpenFace allows for easy integration with other applications and devices through a lightweight messaging system.
View on IEEE
repository.cam.ac.uk
959 Citations
Highly Influential Citations
142
Background Citations
177
Methods Citations
556
Results Citations
8

959 Citations

OpenFace 2.0: Facial Behavior Analysis Toolkit
TLDR
OpenFace 2.0 is an extension of OpenFace toolkit and is capable of more accurate facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Face frontalization for facial expression recognition in the wild
TLDR
A novel Facial Expression-Aware face Frontalization (FEAF) method based on spatial normalization strategy is introduced and a cascade regression model to learn the pair-wise relationship between non-frontal facial shape and its frontal counterpart is developed.
Comparisons of Facial Recognition Algorithms Through a Case Study Application
TLDR
The evaluations of the test cases indicate that among the compared facial recognition algorithms the OpenFace algorithm has the highest accuracy to identify faces and the practitioner and academician are advised on how to improve the accuracy of the current algorithms even further.
REAL-TIME HEAD POSE ESTIMATION WITH SVM MODEL FOR FRONTAL FACE CLASSIFICATION
TLDR
A framework for estimation of the head pose was proposed computing the degrees of freedom (DOF) of the human head using 2D images data only and it was shown that the approach is capable of real-time performance.
Automatic annotation of facial actions from a video record: The case of eyebrows raising and frowning
TLDR
This work proposes a tool which allow to extract eyebrows raising and frowning from the output of two state-of-the-art facial behavior analysis softwares (OpenFace and IntraFace), and suggests that the tool can be used with benefits for automatic annotation purpose.
Hand2Face: Automatic synthesis and recognition of hand over face occlusions
TLDR
A novel framework for synthesizing naturalistic facial occlusions from an initial dataset of non-occluded faces and separate images of hands is introduced, reducing the costly process of data collection and annotation.
OpenMM: An Open-Source Multimodal Feature Extraction Tool
TLDR
The OpenMM is built upon existing open-source repositories to present the first publicly available tool for multimodal feature extraction, and provides a pipeline for researchers to easily extract visual and acoustic features.
3D Head Pose Estimation through Facial Features and Deep Convolutional Neural Networks
TLDR
This paper has developed an end to end face parts segmentation framework through deep convolutional neural networks (DCNNs) for training a deep face parts parsing model, and assess the performance of the newly proposed model on four standard head pose datasets.
Evaluating OpenFace: an open-source automatic facial comparison algorithm for forensics
TLDR
OpenFace in its current form is inadequate for application to forensics, but can be improved to offer promising uses in the field.
Cascade Regression-Based Face Frontalization for Dynamic Facial Expression Analysis
TLDR
The experimental results demonstrate that the proposed cascade regression-based face frontalization (CRFF) method has achieved expression-preserving frontalization, de-occlusion and has improved performance of facial expression recognition.
...
...

References

SHOWING 1-10 OF 80 REFERENCES
Facial Expression Analysis
TLDR
This chapter reviews fundamental approaches to facial measurement by behavioral scientists and current efforts in automated facial expression recognition, and considers challenges, databases available to the research community, approaches to feature detection, tracking, and representation, and both supervised and unsupervised learning.
The first facial expression recognition and analysis challenge
TLDR
This paper presents the first challenge in automatic recognition of facial expressions to be held during the IEEE conference on Face and Gesture recognition 2011, in Santa Barbara, California, and outlines the evaluation protocol, the data used, and the results of a baseline method for the two sub-challenges.
300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge
TLDR
The main goal of this challenge is to compare the performance of different methods on a new-collected dataset using the same evaluation protocol and the same mark-up and hence to develop the first standardized benchmark for facial landmark localization.
Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction.
TLDR
A novel combination of Adaboost and SVM's enhances performance and the outputs of the classifier change smoothly as a function of time, providing a potentially valuable representation to code facial expression dynamics in a fully automatic and unobtrusive manner.
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
TLDR
This paper provides a comprehensive analysis of facial representations by uncovering their advantages and limitations, and elaborate on the type of information they encode and how they deal with the key challenges of illumination variations, registration errors, head-pose variations, occlusions, and identity bias.
Cross-dataset learning and person-specific normalisation for automatic Action Unit detection
TLDR
This paper presents a real-time Facial Action Unit intensity estimation and occurrence detection system based on appearance (Histograms of Oriented Gradients) and geometry features (shape parameters and landmark locations) and demonstrates the generalisability of this approach.
Incremental Face Alignment in the Wild
TLDR
It is shown that it is possible to automatically construct robust discriminative person and imaging condition specific models 'in- the-wild' that outperform state-of-the-art generic face alignment strategies.
Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild
TLDR
The Constrained Local Neural Field model for facial landmark detection is presented, which introduces a probabilistic patch expert (landmark detector) that can learn non-linear and spatial relationships between the input pixels and the probability of a landmark being aligned.
Advances, Challenges, and Opportunities in Automatic Facial Expression Recognition
TLDR
This chapter considers the problem of automatic facial expression analysis, and selects what it believes are the most suitable state-of-the-art algorithms, and a selection of exemplars chosen to characterise a specific approach.
Interactive Facial Feature Localization
TLDR
An improvement to the Active Shape Model is proposed that allows for greater independence among the facial components and improves on the appearance fitting step by introducing a Viterbi optimization process that operates along the facial contours.
...
...