Are You Really Looking at Me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze From Conventional Video

  title={Are You Really Looking at Me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze From Conventional Video},
  author={Minh Tran and Taylan K. Sen and Kurtis Glenn Haut and Mohammad Rafayet Ali and Ehsan Hoque},
  journal={IEEE Transactions on Affective Computing},
Despite a revolution in the pervasiveness of video cameras in our daily lives, one of the most meaningful forms of nonverbal affective communication, interpersonal eye gaze, i.e., eye gaze relative to a conversation partner, is not available from common video. We introduce the Interpersonal-Calibrating Eye-gaze Encoder (ICE), which automatically extracts interpersonal gaze from video recordings without specialized hardware and without prior knowledge of participant locations. Leveraging the… 

Eye Gaze Detection Based on Computational Visual Perception and Facial Landmarks

This work aims to implement eye gaze detection by considering facial landmarks with two different Convolutional Neural Network models, namely the AlexNet model and the VGG16 model, and outperforms the traditionalEye gaze detection system which only uses computer vision and the HAAR classifier in several evaluation metric scores.

Evaluating Calibration-free Webcam-based Eye Tracking for Gaze-based User Modeling

An unsupervised algorithm that maps gaze vectors from a webcam to fixation features used for user modeling, bypassing the need for screen-based gaze coordinates, which require a calibration process, is developed.

Human-Machine Evaluation for Improving Gaze Estimation for Psychiatric, Child Populations and In-the-Wild Videos

  • Psychology
  • 2022
Human annotations are commonly used to improve the performance of machine 1 learning (ML) models. However, previous work has shown that human annotations 2 of gaze lack inter-rater reliability. If

Interpretability by design using computer vision for behavioral sensing in child and adolescent psychiatry

Observation is an essential tool for understand-ing and studying human behavior and mental states. However, coding human behavior is a time-consuming, expensive task, in which reliability can be

Multimodal Driver State Modeling through Unsupervised Learning

‘Can you hear me?’: communication, relationship and ethics in video-based telepsychiatric consultations

There is evidence for ethically relevant changes of the therapeutic relationship in video-based telepsychiatric consultations, which need to be more carefully considered in psychiatric practice and future studies.

Communication Objectives Model (COM): A Taxonomy of Face-to-Face Communication Objectives to Inform Tele-Presence Technology Adoption

The Communication Objectives Model (COM) is reported, a framework developed to understand differences in the performance of communication objectives between CMC and face-to-face interactions, and guide future research on measurement of such communication objectives.



Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery

A novel method for eye contact detection that combines a state-of-the-art appearance-based gaze estimator with a novel approach for unsupervised gaze target discovery, i.e. without the need for tedious and time-consuming manual data annotation is presented.

Robust eye contact detection in natural multi-person interactions using gaze and speaking behaviour

A novel method to robustly detect eye contact in natural three- and four- person interactions using off-the-shelf ambient cameras and exploits that, during conversations, people tend to look at the person who is currently speaking.

Don't Look at Me, I'm Wearing an Eyetracker!

This work investigates what happens to a person's looking behavior when the person with whom they are speaking is also wearing an eye-tracker, and shows that people tend to look less to the eyes of people who are wearing a tracker, than they do to the Eyes of those who are not.

Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes

It is concluded that gaze is an excellent predictor of conversational attention in multiparty conversations and may form a reliable source of input for conversational systems that need to establish whom the user is speaking or listening to.

ROC speak: semi-automated personalized feedback on nonverbal behavior from recorded videos

A framework that couples computer algorithms with human intelligence in order to automatically sense and interpret nonverbal behavior and synthesize Mechanical Turk workers' interpretations, ratings, and comment rankings with the machine-sensed data is presented.

A Look Is Worth a Thousand Words: Full Gaze Awareness in Video-Mediated Conversation

Full gaze awareness, defined here as knowing what someone is looking at, might be expected to be a powerful communicative resource when the conversation concerns some object of common interest in the

Gaze correction for home video conferencing

This work presents a gaze correction approach based on a single Kinect sensor that preserves both the integrity and expressiveness of the face as well as the fidelity of the scene as a whole, producing nearly artifact-free imagery.

Non-Intrusive Gaze Tracking Using Artificial Neural Networks

An empirical analysis of the performance of a large number of artificial neural network architectures for gaze tracking, and Suggestions for further explorations for neurally based gaze trackers are presented, and are related to other similar artificial Neural network applications such as autonomous road following.

Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction

Pupil is an accessible, affordable, and extensible open source platform for pervasive eye tracking and gaze-based interaction and includes state-of-the-art algorithms for real-time pupil detection and tracking, calibration, and accurate gaze estimation.

Real-time eye gaze direction classification using convolutional neural network

  • Anjith GeorgeA. Routray
  • Computer Science
    2016 International Conference on Signal Processing and Communications (SPCOM)
  • 2016
A convolutional neural network is employed in this work for the classification of eye gaze direction and estimation of eye accessing cues and this algorithm was tested on Eye Chimera database and found to outperform state of the art methods.