Exploring Temporal Patterns in Classifying Frustrated and Delighted Smiles

  title={Exploring Temporal Patterns in Classifying Frustrated and Delighted Smiles},
  author={Ehsan Hoque and Daniel J. McDuff and Rosalind W. Picard},
  journal={IEEE Transactions on Affective Computing},
We create two experimental situations to elicit two affective states: frustration, and delight. In the first experiment, participants were asked to recall situations while expressing either delight or frustration, while the second experiment tried to elicit these states naturally through a frustrating experience and through a delightful video. There were two significant differences in the nature of the acted versus natural occurrences of expressions. First, the acted instances were much easier… 

Capturing Laughter and Smiles under Genuine Amusement vs. Negative Emotion

While this work relies on a state-of-the-art smile recognizer, for laughter recognition its transfer learning architecture enhanced on modest data outperforms other models with up to 85% accuracy, suggesting this technique as promising for improving affect models.

A trigger-substrate model for smiling during an automated formative quiz: engagement is the substrate, not frustration

The apparent disconnect between the increased smiling during incorrect answers but the lack of correlation between VAS frustration and smiles suggests a trigger-substrate model where engagement is the permissive substrate, while the noises made by the quiz after wrong answers may be the trigger.

Spontaneous vs. Posed smiles - can we tell the difference?

This work aims to distinguish spontaneous (felt) smile expressions from posed (deliberate) smiles by extracting and analyzing both global motion of the face and subtle changes in the facial expression features through both tracking a series of facial fiducial markers as well as using dense optical flow.

Spontaneous Versus Posed Smiles - Can We Tell the Difference?

This work aims to distinguish spontaneous (felt) smile expressions from posed (deliberate) smiles by extracting and analyzing both global motion of the face and subtle changes in the facial expression features through both tracking a series of facial fiducial markers as well as using dense optical flow.

Predicting online media effectiveness based on smile responses gathered over the Internet

An automated method based on over 1,500 facial responses to media collected over the Internet shows the possibility for an ecologically valid, unobtrusive, evaluation of commercial “liking” and “desire to view again”, strong predictors of marketing success, based only on facial responses.

A Hidden Markov Model Approach to Distinguishing Between Non-Prototypical Displays of Boredom and Interest by Justine Heritage

A Hidden Markov Model approach to distinguishing between naturally occurring instances of two nonprototypical emotions: boredom and interest is presented.

Towards Understanding Confusion and Affective States Under Communication Failures in Voice-Based Human-Machine Interaction

A series of two studies conducted to understand user's affective states during voice-based human-machine interactions, interested in understanding “confusion” in relation with other affectiveStates.

A User Perception--Based Approach to Create Smiling Embodied Conversational Agents

A computational model, based on a corpus of users’ perceptions of smiling and nonsmiling virtual agents, enables a virtual agent to determine the appropriate smiling behavior to adopt given the interpersonal stance it wants to express.

Semantic Similarity of Social Functional Smiles and Laughter

Laughter and smiles co-occur and accomplish similar communicative tasks. Certain smiles and laughter elicit positive affect in the sender and the recipient, serving as social rewards. Other smiles

A Study on the Suppression of Amusement

It was found that arousal induced by amusement could not readily be eliminated, even when the individuals succeeded in masking it on their faces, which further validates the claim in the social psychology literature that suppression decreases expressive behavior, but does not decrease the emotional experience.



Acted vs. natural frustration and delight: Many people smile in natural frustration

The differences in the patterns of smiling that are seen under natural frustration and delight conditions are explored, to see if there might be something measurably different about the smiles in these two cases, which could ultimately improve the performance of classifiers applied to natural expressions.

Social and emotional messages of smiling: An ethological approach.

Did smiling evolve as an expression of happiness, friendliness, or both? Naturalistic observation at a bowling alley (N — 1,793 balls) shows that bowlers often smile when socially engaged, looking at

Coherence between expressive and experiential systems in emotion

Abstract In order to assess the extent of coherence in emotional response systems, we examined the relationship between facial expression and self-report of emotion at multiple points in time during

Sociality of Solitary Smiling: Potentiation by an Implicit Audience

Ss viewed a pleasant videotape either: (a) alone, (b) alone but with the belief that a friend nearby was otherwise engaged, (c) alone but with the belief that a friend was viewing the same videotape

Facial Expressions in Hollywood's Portrayal of Emotion

Much theory and research on emotion are based on the facial expressions of amateurs asked to pose for still photographs. The theory of facial affect programs (FAPs; P. Ekman, 1972) was proposed to

What's in a smile?

This study compared Duchenne and non-Duchenne smiles in early infancy for clues to their emotional significance, suggesting infant smiling does not fit adult models of emotional functioning.

The SEMAINE corpus of emotionally coloured character interactions

A new corpus of emotionally coloured conversations recorded while holding conversations with an operator who adopts in sequence four roles designed to evoke emotional reactions is made available to the scientific community through a web-accessible database.

Building Autonomous Sensitive Artificial Listeners

A fully autonomous integrated real-time system which combines incremental analysis of user behavior, dialogue management, and synthesis of speaker and listener behavior of a SAL character displayed as a virtual agent is described.

Are Spontaneous Expressions and Emotions Linked? an Experimental Test of Coherence

We measured facial behaviors shown by participants in a laboratory study in which a film was used to elicit intense emotions. Participants provided subjective reports of their emotions and their

Automatic, Dimensional and Continuous Emotion Recognition

Recent advances in dimensional and continuous affect modeling, sensing, and automatic recognition from visual, audio, tactile, and brain-wave modalities are explored.