Synthesizing Skeletal Motion and Physiological Signals as a Function of a Virtual Human's Actions and Emotions

  title={Synthesizing Skeletal Motion and Physiological Signals as a Function of a Virtual Human's Actions and Emotions},
  author={Bonny Banerjee and Masoumeh Heidari Kapourchali and Murchana Baruah and Mousumi Deb and Kenneth M. Sakauye and Mette Sofie Olufsen},
Round-the-clock monitoring of human behavior and emotions is required in many healthcare applications which is very expensive but can be automated using machine learning (ML) and sensor technologies. Unfortunately, the lack of infrastructure for collection and sharing of such data is a bottleneck for ML research applied to healthcare. Our goal is to circumvent this bottleneck by simulating a human body in virtual environment. This will allow generation of potentially infinite amounts of… 

Figures and Tables from this paper


TROIKA: A General Framework for Heart Rate Monitoring Using Wrist-Type Photoplethysmographic Signals During Intensive Physical Exercise
A general framework, termed TROIKA, is proposed, which consists of signal decomposiTion for denoising, sparse signal RecOnstructIon for high-resolution spectrum estimation, and spectral peaK trAcking with verification and many variants can be straightforwardly derived from this framework.
State Estimation via Communication for Monitoring
This work shows that limitations of each sensor, such as inference inaccuracy and delay, can be overcome through communication with the other sensors and takes advantage of opportunistic communication by a predictive coding agent for estimating the state of its environment.
Generating 24-hour ECG, BP and respiratory signals with realistic linear and nonlinear clinical characteristics using a nonlinear model
The resultant ECG is shown to exhibit realistic QRSand Qrdispersions, R-S amplitude modulation and Respiratory Sinus Arrhythmia in the short term and normal values for nonlinear statistics (such as entropy) in the long term.
A dynamical model for generating synthetic electrocardiogram signals
A dynamical model based on three coupled ordinary differential equations is introduced which is capable of generating realistic synthetic electrocardiogram (ECG) signals and may be employed to assess biomedical signal processing techniques which are used to compute clinical statistics from the ECG.
A realistic coupled nonlinear artificial ECG, BP, and respiratory signal generator for assessing noise performance of biomedical signal processing algorithms
Extensions to a previously published nonlinear model for generating realistic artificial electrocardiograms to include blood pressure and respiratory signals are presented. The model accurately
A Multimodal Predictive Agent Model for Human Interaction Generation
The proposed attention-based agent is the first of its kind to interact with and learn end-to-end from human interactions, and generate realistic interactions with performance comparable to models without attention and using significantly more computational resources.
Spectral style transfer for human motion between independent actions
This work introduces a spectral style transfer method for human motion between independent actions, thereby greatly reducing the required effort and cost of creating such databases.
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
A multimodal data set for the analysis of human affective states was presented and a novel method for stimuli selection is proposed using retrieval by affective tags from the website, video highlight detection, and an online assessment tool.
How Do We Recognize Emotion From Movement? Specific Motor Components Contribute to the Recognition of Each Emotion
Research-based knowledge regarding the relationship between Laban motor components and bodily emotional expressions can be used by dance-movement and drama therapists for better understanding of clients' emotional movements, for creating appropriate interventions, and for enhancing communication with other practitioners regarding bodily emotional expression.
Dynamic causal modeling of spontaneous fluctuations in skin conductance
A dynamic causal model of how SNA causes spontaneous fluctuations in skin conductance is described, and variational Bayesian model inversion is applied to infer SNA, given empirically observed SF.