Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection

@article{Liu2020RelationMW,
  title={Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection},
  author={Zhilei Liu and Jiahui Dong and Cuicui Zhang and Longbiao Wang and Jianwu Dang},
  journal={ArXiv},
  year={2020},
  volume={abs/1910.10334}
}
Most existing AU detection works considering AU relationships are relying on probabilistic graphical models with manually extracted features. This paper proposes an end-to-end deep learning framework for facial AU detection with graph convolutional network (GCN) for AU relation modeling, which has not been explored before. In particular, AU related regions are extracted firstly, latent representations full of AU information are learned through an auto-encoder. Moreover, each latent… Expand
MER-GCN: Micro-Expression Recognition Based on Relation Modeling with Graph Convolutional Networks
TLDR
This work proposes an end-to-end AU-oriented graph classification network, namely MER-GCN, which uses 3D ConvNets to extract AU features and applies GCN layers to discover the dependency laying between AU nodes for ME categorization and is the first end- to-end architecture for Micro-Expression Recognition (MER) using AUs based GCN. Expand
Facial Action Unit Detection With Transformers
TLDR
The proposed method for detecting Facial Action Units (FAU), which define particular face muscle activity, from an input image, outperforms state-of-theart techniques on two public datasets, BP4D and DISFA, with an absolute improvement of the F1-score of over 2% on each. Expand
Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition
TLDR
This work proposes two auxiliary AU related tasks to bridge the gap between limited annotations and the model performance in a self-supervised manner via the unlabeled data and designs a task of RoI inpainting to recover the randomly cropped AU patches to enhance the discrimination of regional features with AU relation embedding. Expand
AU-assisted Graph Attention Convolutional Network for Micro-Expression Recognition
TLDR
A novel micro-expression recognition approach by combining Action Units (AUs) and emotion category labels is proposed, based on facial muscle movements, which outperforms other state-of-the-art methods on both single database and cross-database micro- expression recognition. Expand
Computational efficient deep neural network with difference attention maps for facial action unit detection
TLDR
A large number of experimental results show that the proposed CEDNN is obviously better than the traditional deep learning method on DISFA+ and CK+ datasets and after adding spatial attention maps, the result is best than the most advanced AU detection method. Expand
GEME: Dual-stream multi-task GEnder-based micro-expression recognition
TLDR
The current study proves that selecting relevant features of micro-expressions distinctive to the gender and added to the micro-expression features improves themicro-expression recognition accuracy. Expand
Deep graph learning for semi-supervised classification
TLDR
DGL can not only learn the global structure by the previous layer metric computation updating, but also mine the local structure by next layer local weight reassignment, and deeply mine the relationship of the different structures by the hierarchical progressive learning for improving the performance of semi-supervised classification. Expand
An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge
TLDR
To mitigate the problem of limited and biased ME data, synthetic data generation is surveyed for the diversity enrichment of micro-expression data and the state-of-the-art spotting works are introduced. Expand
Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
TLDR
This survey thoroughly review the different types of graph architectures and their applications in healthcare in a systematic manner, organized by their domain of application including functional connectivity, anatomical structure, and electrical-based analysis. Expand
CaFGraph: Context-aware Facial Multi-graph Representation for Facial Action Unit Recognition
  • Yingjie Chen, Diqi Chen, Yizhou Wang, Tao Wang, Yun Liang
  • Computer Science
  • ACM Multimedia
  • 2021
TLDR
CaFGraph is the first work to construct a universal facial multi-graph structure that is independent of both task settings and dataset statistics for almost all fine-grained facial behavior analysis tasks, including but not limited to AU recognition. Expand
...
1
2
...

References

SHOWING 1-10 OF 32 REFERENCES
Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition
TLDR
This paper investigates how to integrate the semantic relationship propagation between AUs in a deep neural network framework to enhance the feature representation of facial regions, and proposes an AU semantic relationship embedded representation learning (SRERL) framework. Expand
Capturing Global Semantic Relationships for Facial Action Unit Recognition
TLDR
A hierarchical model is built that combines the bottom-level image features and the top-level AU relationships to jointly recognize AUs in a principled manner and can successfully capture them to more accurately characterize the AU relationships. Expand
Deep Region and Multi-label Learning for Facial Action Unit Detection
TLDR
This paper proposes Deep Region and Multi-label Learning (DRML), a unified deep network that simultaneously addresses these two problems of facial Action Unit detection and region learning, allowing the two seemingly irrelevant problems to interact more directly. Expand
Facial Action Unit Recognition and Intensity Estimation Enhanced Through Label Dependencies
TLDR
The results demonstrate that the proposed approaches faithfully model the complex and global inherent AU dependencies, and the expression labels available only during training can boost the estimation of AU dependencies for both AU recognition and intensity estimation. Expand
Joint Patch and Multi-label Learning for Facial Action Unit and Holistic Expression Recognition
TLDR
A joint patch and multi-label learning (JPML) framework that models the structured joint dependence behind features, AUs, and their interplay, and can be extended to recognize holistic expressions by learning common and specific patches, which afford a more compact representation than the standard expression recognition methods. Expand
Conditional Adversarial Synthesis of 3D Facial Action Units
TLDR
This paper proposes an AU synthesis framework that combines the well-known 3D Morphable Model (3DMM), which intrinsically disentangles expression parameters from other face attributes, with models that adversarially generate 3DMM expression parameters conditioned on given target AU labels, in contrast to the more conventional approach of generating facial images directly. Expand
EAC-Net: A Region-Based Deep Enhancing and Cropping Approach for Facial Action Unit Detection
TLDR
This paper proposes a deep learning based approach for facial action unit detection by enhancing and cropping the regions of interest by adding two novel nets: the enhancing layers and the cropping layers, to a pretrained CNN model. Expand
Exploiting sparsity and co-occurrence structure for action unit recognition
TLDR
A novel Bayesian framework for facial action unit recognition that encodes these two natural aspects of facial action units via compressed sensing and group-wise sparsity inducing priors and derives an efficient inference scheme that can be automatically learned from data. Expand
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
TLDR
This work revisits both the alignment step and the representation step by employing explicit 3D face modeling in order to apply a piecewise affine transformation, and derive a face representation from a nine-layer deep neural network. Expand
DISFA: A Spontaneous Facial Action Intensity Database
TLDR
To meet the need for publicly available corpora of well-labeled video, the Denver intensity of spontaneous facial action database is collected, ground-truthed, and prepared for distribution. Expand
...
1
2
3
4
...