• Corpus ID: 211572443

Efficiently Guiding Imitation Learning Algorithms with Human Gaze

@article{Saran2020EfficientlyGI,
  title={Efficiently Guiding Imitation Learning Algorithms with Human Gaze},
  author={Akanksha Saran and Ruohan Zhang and Elaine Schaertl Short and Scott Niekum},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.12500}
}
Human gaze is known to be an intention-revealing signal in human demonstrations of tasks. In this work, we use gaze cues from human demonstrators to enhance the performance of state-of-the-art inverse reinforcement learning (IRL) and behavior cloning (BC) algorithms, without adding any additional learnable parameters to those models. We show how to augment any existing convolutional architecture with our auxiliary gaze loss (coverage-based gaze loss or CGL) that can guide learning toward a… 

Figures and Tables from this paper

Human versus Machine Attention in Deep Reinforcement Learning Tasks
TLDR
Light is shed on the inner workings of deep reinforcement learning models by analyzing the pixels that they attend to during task execution, and comparing them with the pixels attended to by humans executing the same tasks.
Machine versus Human Attention in Deep Reinforcement Learning Tasks
TLDR
Light is shed on the inner workings of deep reinforcement learning models by analyzing the pixels that they attend to during task execution, and comparing them with the pixels attended to by humans executing the same tasks.
Self-Supervised Attention-Aware Reinforcement Learning
TLDR
This work proposes a novel self-supervised attention learning approach which can learn to select regions of interest without explicit annotations, and act as a plug for existing deep RL methods to improve the learning performance.
Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation
TLDR
This paper focuses on the task of learning from feedback, in which the human trainer not only gives binary evaluative “good" or “bad" feedback for queried state-action pairs, but also provides a visual explanation by annotating relevant features in images.
Improving Reinforcement Learning with Human Assistance: An Argument for Human Subject Studies with HIPPO Gym
TLDR
A new open-source RL framework, the Human Input Parsing Platform for Openai Gym (HIPPO Gym), and the design decisions that went into its creation are introduced, to facilitate human-RL research.
Recent advances in leveraging human guidance for sequential decision-making tasks
TLDR
This survey provides a high-level overview of five recent machine learning frameworks that primarily rely on human guidance apart from pre-specified reward functions or conventional, step-by-step action demonstrations.
Understanding Human Intelligence in the Era of Artificial Intelligence
TLDR
This research explores how human intelligence and machine intelligence are related to each other intending to build human-centered intelligent machines and aims to discover the mechanism of intelligence by studying human brains and behaviors and applying these insights to the field of AI.
Using Physiological Measurements to Analyze the Tactical Decisions in Human Swarm Teams
TLDR
Mental engagement and Fz-O1 coherence are the important factors in predicting the tactical decisions in a simulated search and rescue mission.

References

SHOWING 1-10 OF 64 REFERENCES
AGIL: Learning Attention from Human for Visuomotor Tasks
TLDR
This work proposes the AGIL (Attention Guided Imitation Learning) framework, a deep neural network that can predict human gaze positions and visual attention with high accuracy and significantly improves the action prediction accuracy and task performance.
Gaze Training by Modulated Dropout Improves Imitation Learning
TLDR
This work proposes a method for integrating gaze information into a deep driving network implicitly rather than as an additional input, and demonstrates that gaze-modulated dropout enhances the generalization capability of the network to unseen scenes.
Understanding Teacher Gaze Patterns for Robot Learning
TLDR
This work studies gaze patterns of human teachers demonstrating tasks to robots and proposes ways in which such patterns can be used to enhance robot learning and provides a foundation for a model of natural human gaze in robot learning from demonstration settings.
A gaze model improves autonomous driving
TLDR
A conditional generative adversarial network is trained that accurately predicts human gaze maps while driving in both familiar and unseen environments and incorporated the predicted gaze maps into end-to-end networks for two behaviors: following and overtaking.
Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset
TLDR
This work provides a large-scale, high-quality dataset of human actions with simultaneously recorded eye movements while humans play Atari video games, and introduces a novel form of gameplay, in which the human plays in a semi-frame-by-frame manner.
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations
TLDR
A novel reward-learning-from-observation algorithm, Trajectory-ranked Reward EXtrapolation (T-REX), that extrapolates beyond a set of (approximately) ranked demonstrations in order to infer high-quality reward functions from a setof potentially poor demonstrations.
Behavioral Cloning from Observation
TLDR
This work proposes a two-phase, autonomous imitation learning technique called behavioral cloning from observation (BCO), that allows the agent to acquire experience in a self-supervised fashion to develop a model which is then utilized to learn a particular task by observing an expert perform that task without the knowledge of the specific actions taken.
Leveraging Human Guidance for Deep Reinforcement Learning Tasks
TLDR
This survey provides a high-level overview of five recent learning frameworks that primarily rely on human guidance other than conventional, step-by-step action demonstrations and reviews the motivation, assumption, and implementation of each framework.
Playing hard exploration games by watching YouTube
TLDR
A two-stage method of one-shot imitation that allows an agent to convincingly exceed human-level performance on the infamously hard exploration games Montezuma's Revenge, Pitfall! and Private Eye for the first time, even if the agent is not presented with any environment rewards.
Periphery-Fovea Multi-Resolution Driving Model Guided by Human Attention
TLDR
It is shown that adding high-resolution input from predicted human driver gaze locations significantly improves the driving accuracy of the model and achieves a significantly higher performance gain in pedestrian-involved critical situations than in other non-critical situations.
...
...