Enhancing Embodied Conversational Agents with Social and Emotional Capabilities

  title={Enhancing Embodied Conversational Agents with Social and Emotional Capabilities},
  author={Bart van Straalen and Dirk K. J. Heylen and Mari{\"e}t Theune and Anton Nijholt},
In this paper we present our current work on an embodied conversational agent for training medical bad news conversations and discuss the inspiration gained from previous work of our own and others. Central in this research is the influence of emotional and social features on the selection and realization of conversational behavior. 

A Conversational Agent Framework with Multi-modal Personality Expression

Consistently exhibited personalities are crucial elements of realistic, engaging, and behavior-rich conversational virtual agents. Both nonverbal and verbal cues help convey these agents’ unseen

How emotions affect relations between interlocutors in bad news conversations

  • B. V. StraalenD. Heylen
  • Computer Science
    2012 5th International Symposium on Communications, Control and Signal Processing
  • 2012
A prototype of dialogue system that allows one to practice holding bad news conversations and insights on how computational models of dialogue can take features of emotions models into account are given.

Designing emotional BDI agents: good practices and open questions

A review of the most relevant proposals to include emotional aspects in the design of B DI agents is presented and common findings and good practices modeling affect, empathy and regulatory capacities in BDI agents are extracted.

Computational Study of Primitive Emotional Contagion in Dyadic Interactions

A cross-recurrence based methodology that explicitly integrates short and long-term temporal dynamics through the analysis of both facial expressions and sentiment was developed and is able to show the importance of the adoption of cross-modal strategies for addressing emotional contagion.

ABC-EBDI: An affective framework for BDI agents

Applying a Psychotherapeutic Theory to the Modeling of Affective Intelligent Agents

The application of a well-known psychotherapeutic model, the Ellis’s ABC theory, to the modeling of intelligent agents is proposed to be able to simulate human realistic behavior in situations in which varied reactions are possible and are determined by several factors such as personality, affective state, prior experiences, and regulation capabilities.

iTutorials for the Aid to Mild Cognitively Impaired Elderly Population in their Preferred Environment

Empiric results obtained while testing with real Portuguese older volunteers demonstrated that the improvement and greater adaptability of iTutorials is easy to learn and pleasant to use, and proved its aid to elderly users with cognitive disabilities in the accomplishment of basic tasks.

Modeling Artificial Personalities through the Expression of Emotions in Narrative Games

Personality modeling is important in order to create character variation in games. Character variation favors replayability and is an important aspect of game design. The effect of artificial

Sentiment Analysis: From Opinion Mining to Human-Agent Interaction

A comparative state of the art which analyzes the sentiment-related phenomena and the sentiment detection methods used in both communities and makes an overview of the goals of socio-affective human-agent strategies is presented.

Modelling a conversational agent (Botocrates) for promoting critical thinking and argumentation skills

Students in higher education institutions are often advised to think critically, yet without being guided to do so. The study investigated the use of a conversational agent (Botocrates) for



Social agents: The first generations

This paper reviews some of the approaches that have been taken in the first decade of ECA research, by presenting the social signaling skills of three agents the authors have developed in their group.

The affective reasoner: a process model of emotions in a multi-agent system

This dissertation has focused on implementing a platform for reasoning about emotions that supports the testing of psychological theory and model a multi-agent world and gives simple affective life to agents in the form of rudimentary emotions and emotion-induced actions.

Affective Dialogue Systems

These difficulties are exemplified on the basis of SympaFly, a database with dialogues between users and a fully automatic speech dialogue telephone system for flight reservation and booking.

Tears and fears: modeling emotions and emotional behaviors in synthetic agents

This work integrates two complementary approaches to emotional modeling into a single unified system to create general computational models to support characters that act in virtual environments, make decisions, but whose behavior also suggests an underlying emotional current.

Emotion based Agent Architectures for Tutoring Systems: The INES Architecture

First it is shown how a more advanced emotion generation architecture, SHAME, can be integrated in INES, and how INES can be extended with an action selection architecture for emotional agents.

FearNot! - An Experiment in Emergent Narrative

We discuss the experience of constructing the application FearNot! (Fun with Empathic Agents Reaching Novel Outcomes in Teaching), an application of virtual drama to anti-bullying education inspired

EMA: A computational model of appraisal dynamics

This work argues for a single and automatic appraisal process that operates over a person’’s interpretation of their relationship to the environment, and illustrates this perspective through the computational modeling of a naturalistic emotional situation.

Generating Socially Appropriate Tutorial Dialog

A dialog generator for pedagogical agents that takes motivation and face threat factors into account enables the agent to interact with learners in a socially appropriate fashion, and foster intrinsic motivation on the part of the learner, which in turn may lead to more positive learner affective states.


INES, an intelligent, multimodal tutoring environment, and how it builds a tutor agent in the environment that tries to be sensitive to the mental state of the student that interacts with it.

Engagement vs. Deceit: Virtual Humans with Human Autobiographies

A virtual human exercise counselor that interacts with users daily to promote exercise, and the integration of a dynamic social storytelling engine used to maintain user engagement with the agent and retention in the intervention are described.