Regression-based intensity estimation of facial action units

Abstract

a r t i c l e i n f o Facial Action Coding System (FACS) is the de facto standard in the analysis of facial expressions. FACS describes expressions in terms of the configuration and strength of atomic units called Action Units: AUs. FACS defines 44 AUs and each AU intensity is defined on a nonlinear scale of five grades. There has been significant progress in the literature on the detection of AUs. However, the companion problem of estimating the AU strengths has not been much investigated. In this work we propose a novel AU intensity estimation scheme applied to 2D luminance and/or 3D surface geometry images. Our scheme is based on regression of selected image features. These features are either non-specific, that is, those inherited from the AU detection algorithm, or are specific in that they are selected for the sole purpose of intensity estimation. For thoroughness , various types of local 3D shape indicators have been considered, such as mean curvature, Gaussian curvature , shape index and curvedness, as well as their fusion. The feature selection from the initial plethora of Gabor moments is instrumented via a regression that optimizes the AU intensity predictions. Our AU intensity estimator is person-independent and when tested on 25 AUs that appear singly or in various combinations , it performs significantly better than the state-of-the-art method which is based on the margins of SVMs designed for AU detection. When evaluated comparatively, one can see that the 2D and 3D modalities have relative merits per upper face and lower face AUs, respectively, and that there is an overall improvement if 2D and 3D intensity estimations are used in fusion. Automated measurement of facial actions has many potential applications for intelligent human–computer interfaces (HCI) and in behavioural science. As discussed thoroughly in ref. [1], computer perception of emotional states can be useful for HCI in various ways, and these states can be inferred via the extracted facial actions. As an application example consider an automatic tutoring system: If the tutor is capable of understanding whether the user is bored or irritated, it can switch to a more affective response mode. Automatic monitoring of operator alertness and of driver fatigue are two other practical domains. Several applications may be envisioned in the entertainment area, from computer games to dolls which respond according to the player's mood. Automatic facial expression recognizers can also be useful …

DOI: 10.1016/j.imavis.2011.11.008

Extracted Key Phrases

11 Figures and Tables

01020201220132014201520162017
Citations per Year

92 Citations

Semantic Scholar estimates that this publication has 92 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@article{Savran2012RegressionbasedIE, title={Regression-based intensity estimation of facial action units}, author={Arman Savran and B{\"{u}lent Sankur and M. Taha Bilge}, journal={Image Vision Comput.}, year={2012}, volume={30}, pages={774-784} }