Predicting the Driver's Focus of Attention: The DR(eye)VE Project

@article{Palazzi2019PredictingTD,
  title={Predicting the Driver's Focus of Attention: The DR(eye)VE Project},
  author={Andrea Palazzi and Davide Abati and Simone Calderara and Francesco Solera and Rita Cucchiara},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2019},
  volume={41},
  pages={1720-1733}
}
In this work we aim to predict the driver's focus of attention. [...] Key Method We also introduce DR(eye)VE, the largest dataset of driving scenes for which eye-tracking annotations are available. This dataset features more than 500,000 registered frames, matching ego-centric views (from glasses worn by drivers) and car-centric views (from roof-mounted camera), further enriched by other sensors measurements. Results highlight that several attention patterns are shared across drivers and can be reproduced to…Expand
Driver's Eye Fixation Prediction by Deep Neural Network
TLDR
This work proposes convolution neural networks to predict the potential saliency regions in the driving environment, and uses the probability of the driver gaze direction, given head pose as a top-down factor to estimate a driver’s eye fixation. Expand
A Driver’s Visual Attention Prediction Using Optical Flow
TLDR
A deep neural network framework is developed that provides attention locations and attention levels using optical flow maps, which represent the movements of contents in videos, and confirms the hypothesis that motion plays a role in prediction accuracy improvement, and there is a margin for accuracy improvement by using motion features. Expand
GG-Net: Gaze Guided Network for Self-driving Cars
Imitation learning is used massively in autonomous driving for training networks to predict steering commands from frames using annotated data collected by an expert driver. Believing that the framesExpand
How Do Drivers Allocate Their Potential Attention? Driving Fixation Prediction via Convolutional Neural Networks
TLDR
The experimental results indicate that the proposed CDNN outperforms the state-of-the-art saliency models and predicts drivers’ attentional locations more accurately, and shows excellent detection of secondary important information or regions that cannot be ignored during driving if they exist. Expand
Real-Time Driver’s Focus of Attention Extraction and Prediction using Deep Learning
Driving is one of the most common activities in our modern lives. Every day, millions drive to and from their schools or workplaces. Even though this activity seems simple and everyone knows how toExpand
“Looking at the Right Stuff” – Guided Semantic-Gaze for Autonomous Driving
TLDR
A novel Semantics Augmented GazE (SAGE) detection approach that captures driving specific contextual information, in addition to the raw gaze, that serves as a powerful tool to focus on the relevant regions in an image frame in order to make driving both safe and efficient. Expand
DGAZE: Driver Gaze Mapping on Road
TLDR
I-DGAZE, a fused convolutional neural network for predicting driver gaze on the road, which was trained on the DGAZE dataset and combines facial features such as face location and head pose along with the image of the left eye to get optimum results is presented. Expand
Prediction of Driver’s Attention Points Based on Attention Model
TLDR
The results show that the DVAN can quickly locate and identify the target that the driver is most interested in a picture, and the average accuracy of prediction is 96.3%. Expand
An Efficient Model for Driving Focus of Attention Prediction using Deep Learning
TLDR
A Y-shape-structured Fully Convolutional Neural Network is proposed to learn and predict the FoA, which can predict the driver’s Focus of Attention with high accuracy at a speed of about 122 frames per second, which outperforms some previous works for FoA prediction. Expand
Training a network to attend like human drivers saves it from common but misleading loss functions
TLDR
A novel FCN-ConvLSTM model to predict multi-focal human driver's attention merely from monocular dash camera videos is proposed and a way of using human gaze to determine the sampling weights is introduced. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 93 REFERENCES
DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving
TLDR
A novel and publicly available dataset acquired during actual driving that contains drivers' gaze fixations and their temporal integration providing task-specific saliency maps and can foster new discussions on better understanding, exploiting and reproducing the driver's attention process in the autonomous and assisted cars of future generations. Expand
Learning where to attend like a human driver
TLDR
This paper model the driver's gaze by training a coarse-to-fine convolutional network on short sequences extracted from the DR(eye)VE dataset and advocates for a new assisted driving paradigm which suggests to the driver, with no intervention, where she should focus her attention. Expand
Where Should You Attend While Driving?
TLDR
This paper models the driver’s gaze by training a coarse-to-fine convolutional network on short sequences extracted from the DR(eye)VE dataset and advocates for a new assisted driving paradigm which suggests to the driver where she should focus her attention. Expand
Driver Gaze Region Estimation Without Using Eye Movement
TLDR
A system that extracts facial features and classifies their spatial configuration into six regions in real-time and achieves an average accuracy of 91.4% at an average decision rate of 11 Hz on a dataset of 50 drivers from an on-road study is proposed. Expand
Driver Gaze Estimation Without Using Eye Movement
TLDR
A system that extracts facial features and classifies their spatial configuration into six regions in real-time and achieves an average accuracy of 91.4% at an average decision rate of 11 Hz on a dataset of 50 drivers from an on-road study is proposed. Expand
Driver Gaze Region Estimation without Use of Eye Movement
TLDR
A proposed system extracts facial features and classifies their spatial configuration into six regions in real time and achieves an average accuracy of 91.4 percent at an average decision rate of 11 Hz on a dataset of 50 drivers from an on-road study. Expand
Robust and continuous estimation of driver gaze zone by dynamic analysis of multiple face videos
TLDR
This paper presents a distributed camera framework for gaze zone estimation using head pose dynamics to operate robustly and continuously even during large head movements, andalyses show that dynamic information significantly improves the results. Expand
Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition
TLDR
This work complements existing state-of-the art large scale dynamic computer vision annotated datasets like Hollywood-2 and UCF Sports with human eye movements collected under the ecological constraints of visual action and scene context recognition tasks, and introduces novel dynamic consistency and alignment measures, which underline the remarkable stability of patterns of visual search among subjects. Expand
Recurrent Mixture Density Network for Spatiotemporal Visual Attention
TLDR
A spatiotemporal attentional model that learns where to look in a video directly from human fixation data, and is optimized via maximum likelihood estimation using human fixations as training data, without knowledge of the action in each video. Expand
Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model
TLDR
This paper presents a novel model which can predict accurate saliency maps by incorporating neural attentive mechanisms, and shows, through an extensive evaluation, that the proposed architecture outperforms the current state-of-the-art on public saliency prediction datasets. Expand
...
1
2
3
4
5
...