• Corpus ID: 231740522

Matching Representations of Explainable Artificial Intelligence and Eye Gaze for Human-Machine Interaction

  title={Matching Representations of Explainable Artificial Intelligence and Eye Gaze for Human-Machine Interaction},
  author={Tiffany Hwu and Mia Levy and Steven Skorheim and David J. Huber},
Rapid non-verbal communication of task-based stimuli is a challenge in human-machine teaming, particularly in closed-loop interactions such as driving. To achieve this, we must understand the representations of information for both the human and machine, and determine a basis for bridging these representations. Techniques of explainable artificial intelligence (XAI) such as layer-wise relevance propagation (LRP) provide visual heatmap explanations for high-dimensional machine learning… 

Figures and Tables from this paper


SALICON: Saliency in Context
A mouse-contingent multi-resolutional paradigm based on neurophysiological and psychophysical studies of peripheral vision, to simulate the natural viewing behavior of humans is designed, thus enabling large-scale data collection.
DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving
A novel and publicly available dataset acquired during actual driving that contains drivers' gaze fixations and their temporal integration providing task-specific saliency maps and can foster new discussions on better understanding, exploiting and reproducing the driver's attention process in the autonomous and assisted cars of future generations.
Understanding Low- and High-Level Contributions to Fixation Prediction
Comparing different features within the same powerful readout architecture allows to better understand the relevance of low- versus high-level features in predicting fixation locations, while simultaneously achieving state-of-the-art saliency prediction.
Predicting the Driver's Focus of Attention: The DR(eye)VE Project
This work proposes a new computer vision model based on a multi-branch deep architecture that integrates three sources of information: raw video, motion and scene semantics, and introduces DR(eye)VE, the largest dataset of driving scenes for which eye-tracking annotations are available.
Boosting bottom-up and top-down visual features for saliency estimation
  • A. Borji
  • Computer Science
    2012 IEEE Conference on Computer Vision and Pattern Recognition
  • 2012
The boosting model outperforms 27 state-of-the-art models and is so far the closest model to the accuracy of human model for fixation prediction, and successfully detects the most salient object in a scene without sophisticated image processings such as region segmentation.
Predicting Driver Attention in Critical Situations
A new in-lab driver attention collection protocol is proposed and a new driver attention dataset is introduced, Berkeley DeepDrive Attention (BDD-A) dataset, which is built upon braking event videos selected from a large-scale, crowd-sourced driving video dataset.
CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research
This work records eye movements of 120 observers while they freely viewed a large number of naturalistic and artificial images, which opens new challenges for the next generation of saliency models and helps conduct behavioral studies on bottom-up visual attention.
Relevance-Based Data Masking: A Model-Agnostic Transfer Learning Approach for Facial Expression Recognition
This work introduces a novel approach to transfer learning, which addresses two shortcomings of traditional methods: The (partial) inheritance of the original models structure and the restriction to other neural network models as an input source.
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
This work proposes a general solution to the problem of understanding classification decisions by pixel-wise decomposition of nonlinear classifiers by introducing a methodology that allows to visualize the contributions of single pixels to predictions for kernel-based classifiers over Bag of Words features and for multilayered neural networks.
iNNvestigate neural networks!
The presented library iNNvestigate addresses this by providing a common interface and out-of-the- box implementation for many analysis methods, including the reference implementation for PatternNet and PatternAttribution as well as for LRP-methods.