Frame Attention Networks for Facial Expression Recognition in Videos

@article{Meng2019FrameAN,
  title={Frame Attention Networks for Facial Expression Recognition in Videos},
  author={Debin Meng and Xiaojiang Peng and Kai Wang and Yu Qiao},
  journal={2019 IEEE International Conference on Image Processing (ICIP)},
  year={2019},
  pages={3866-3870}
}
The video-based facial expression recognition aims to classify a given video into several basic emotions. [...] Key Method The whole network is composed of two modules. The feature embedding module is a deep Convolutional Neural Network (CNN) which embeds face images into feature vectors. The frame attention module learns multiple attention weights which are used to adaptively aggregate the feature vectors to form a single discriminative video representation. We conduct extensive experiments on CK+ and AFEW8.0…Expand
Hybrid Attention Cascade Network for Facial Expression Recognition
TLDR
A convenient facial expression recognition cascade network comprising spatial feature extraction, hybrid attention, and temporal feature extraction that achieves not only competitive performance comparable to state-of-the-art methods but also greater than 2% performance improvement on the AFEW dataset, proving the significant outperformance of facial expressions recognition in the natural environment. Expand
Two-Stream Attention Network for Pain Recognition from Video Sequences
TLDR
An end-to-end approach based on attention networks for the analysis and recognition of pain-related facial expressions is proposed, which combines both spatial and temporal aspects of facial expressions through a weighted aggregation of attention-based neural networks’ outputs. Expand
Lossless Attention in Convolutional Networks for Facial Expression Recognition in the Wild
TLDR
A Lossless Attention Model (LLAM) for convolutional neural networks (CNN) to extract attention-aware features from faces to avoid decay information in the process of generating attention maps by using the information of the previous layer and not reducing the dimensionality. Expand
Identity-aware Facial Expression Recognition in Compressed Video
TLDR
This paper targets to explore the inter-subject variations eliminated facial expression representation in the compressed video domain and can achieve comparable or better performance than the recent decoded image based methods on the typical FER benchmarks with about 3× faster inference with compressed data. Expand
Self-Difference Convolutional Neural Network for Facial Expression Recognition
TLDR
A self-difference convolutional network (SD-CNN) is proposed to address the intra-class variation issue in FER and achieves state-of-the-art performance with accuracies of 99.7% on CK+ and 91.3% on Oulu-CASIA, respectively. Expand
Temporal Stochastic Softmax for 3D CNNs: An Application in Facial Expression Recognition
TLDR
The proposed softmax strategy provides several advantages – a reduced computational complexity due to efficient clip sampling, and an improved accuracy since temporal weighting focuses on more relevant clips during both training and inference. Expand
Facial Expression Recognition with Grid-Wise Attention and Visual Transformer
  • Qionghao Huang, Changqin Huang, Xizhe Wang, Fan Jiang
  • Computer Science
  • 2021
TLDR
A novel FER framework with two attention mechanisms for CNN-based models are introduced, and these two Attention mechanisms are used for the low-level feature learning the high-level semantic representation and the global representation, respectively. Expand
Multimodal(Audio, Facial and Gesture) based Emotion Recognition challenge
  • Wei Gou, Jian Li, Mo Sun
  • Computer Science
  • 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)
  • 2020
TLDR
This work expands the focus to several expression forms to facilitate research on emotion recognition in the wild and shows that the overall accuracy of the algorithm on the validation dataset of the challenge is 76.43%. Expand
An Efficacious Method for Facial Expression Recognition: GAN Erased Facial Feature Network (GE2FN)
  • Tao Zhang, Tang Kai
  • Computer Science
  • ICMLC
  • 2021
TLDR
A powerful facial expression recognition method based on removing the noise features from the input image to achieve a significant improvement in accuracy, and the accuracy rate is significantly improved compared to other approaches, including the current mainstream VGG approach. Expand
3D-CNN for Facial Emotion Recognition in Videos
TLDR
It is shown that 3D convo-lutional neural networks (3D-CNN) can be very good for predicting facial emotions that are expressed over a sequence of frames, and the proposed resulting architecture improves over the results of the state-of-the-art techniques when tested on the CK+ and Oulu-CASIA datasets. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 31 REFERENCES
Neural Aggregation Network for Video Face Recognition
TLDR
This NAN is trained with a standard classification or verification loss without any extra supervision signal, and it is found that it automatically learns to advocate high-quality face images while repelling low-quality ones such as blurred, occluded and improperly exposed faces. Expand
Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks
TLDR
This paper proposes a part-based hierarchical bidirectional recurrent neural network (PHRNN) to analyze the facial expression information of temporal sequences and reduces the error rates of the previous best ones on three widely used facial expression databases. Expand
Video-based Emotion Recognition Using Deeply-Supervised Neural Networks
TLDR
A novel deeply supervised CNN (DSN) architecture is developed, taking the multi-level and multi-scale features extracted from different convolutional layers to provide a more advanced representation of ER. Expand
Combining modality specific deep neural networks for emotion recognition in video
In this paper we present the techniques used for the University of Montréal's team submissions to the 2013 Emotion Recognition in the Wild Challenge. The challenge is to classify the emotionsExpand
Multi-cue fusion for emotion recognition in the wild
TLDR
This work proposes a multi-cue fusion emotion recognition (MCFER) framework by modeling human emotions from three complementary cues, i.e., facial texture, facial landmark action and audio signal, and then fusing them together. Expand
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
TLDR
A deep learning technique, which is regarded as a tool to automatically extract useful features from raw data, is adopted and is combined using a new integration method in order to boost the performance of the facial expression recognition. Expand
A Compact Deep Learning Model for Robust Facial Expression Recognition
TLDR
The proposed framework is extended to a frame-to-sequence approach by exploiting temporal information with gated recurrent units and an illumination augmentation scheme to alleviate the overfitting problem when training the deep networks with hybrid data sources. Expand
Video-based emotion recognition using CNN-RNN and C3D hybrid networks
TLDR
Extensive experiments show that combining RNN and C3D together can improve video-based emotion recognition noticeably, and are presented to the EmotiW 2016 Challenge. Expand
Temporal multimodal fusion for video emotion classification in the wild
TLDR
Improved face descriptors based on 2D and 3D Convolutional Neural Networks are proposed and several fusion methods, temporal and multimodal, including a novel hierarchical method combining features and scores are explored. Expand
Island Loss for Learning Discriminative Features in Facial Expression Recognition
TLDR
Experimental results on four benchmark expression databases have demonstrated that the CNN with the proposed island loss (IL-CNN) outperforms the baseline CNN models with either traditional softmax loss or center loss and achieves comparable or better performance compared with the state-of-the-art methods for facial expression recognition. Expand
...
1
2
3
4
...