• Corpus ID: 25985931

Saliency Benchmarking: Separating Models, Maps and Metrics

  title={Saliency Benchmarking: Separating Models, Maps and Metrics},
  author={Matthias K{\"u}mmerer and Thomas S. A. Wallis and Matthias Bethge},
The field of fixation prediction is heavily model-driven, with dozens of new models published every year. [] Key Method We define a saliency model to be a probabilistic model of fixation density prediction and, inspired by Bayesian decision theory, a saliency map to be a metric-specific prediction derived from the model density which maximizes the expected performance on that metric.

Figures from this paper

Faster gaze prediction with dense networks and Fisher pruning

Through a combination of knowledge distillation and Fisher pruning, this paper obtains much more runtime-efficient architectures for saliency prediction, achieving a 10x speedup for the same AUC performance as a state of the art network on the CAT2000 dataset.

Learning Visual Importance for Graphic Designs and Data Visualizations

An automated models that predict the relative importance of different elements in data visualizations and graphic designs and how such predictions of importance can be used for automatic design retargeting and thumbnailing are presented.

A dynamical scan path model for task-dependence during scene viewing

This work proposes a scan path model for both fixation positions and fixation durations, which includes influences of task instructions and interindividual differences, and results demonstrate that model properties such as the attentional span are adjusted to task requirements.

Guiding human gaze with convolutional neural networks

A new model for manipulating images to change the distribution of human fixations in a controlled fashion is presented, using the state-of-the-art model for fixation prediction to train a convolutional neural network to transform images so that they satisfy a given fixation distribution.

Introducing UN Salient360! Benchmark: A platform for evaluating visual attention models for 360° contents

This paper introduces the ‘UN Salient360! benchmark’ platform featuring a dataset, a toolbox and a framework for evaluation of different class of models, for evaluating and comparing the performance of models for saliency and scanpath prediction for 360° content.

Improving unsupervised saliency detection by migrating from RGB to multispectral images

The main aim of this study is to investigate the advantages of using multispectral images instead of RGB images for saliency detection within the framework of unsupervised models, and shows a general improvement when mult ispectral information is taken into account.

Disentangling bottom-up versus top-down and low-level versus high-level influences on eye movements over time.

The results lend support to a separation of visual scene exploration into three phases: the first saccade, an initial guided exploration characterized by a gradual broadening of the fixation density, and a steady state that is reached after roughly 10 fixations.

Community-based benchmarking improves spike inference from two-photon calcium imaging data

The results of the spikefinder challenge, launched to catalyze the development of new spike inference algorithms through crowd-sourcing, are reported, and ten of the submitted algorithms are presented which show improved performance compared to previously evaluated methods.

Community-based benchmarking improves spike rate inference from two-photon calcium imaging data

The results of the spikefinder challenge, launched to catalyze the development of new spike rate inference algorithms through crowd-sourcing, are reported, and the top-performing algorithms are based on a wide range of principles, yet provide highly correlated estimates of the neural activity.

Computational perception for multi-modal document understanding

The crowdsourced attention data captured from novel user interfaces is used to train neural network models to predict where people look in graphic designs and information visualizations, with demonstrated applications to thumbnailing, design retargeting, and interactive feedback within graphic design tools.



Information-theoretic model comparison unifies saliency metrics

This work brings saliency evaluation into the domain of information by framing fixation prediction models probabilistically and calculating information gain, and jointly optimize the scale, the center bias, and spatial blurring of all models within this framework.

What Do Different Evaluation Metrics Tell Us About Saliency Models?

This paper provides an analysis of 8 different evaluation metrics and their properties, and makes recommendations for metric selections under specific assumptions and for specific applications.

Saliency and Human Fixations: State-of-the-Art and Study of Comparison Metrics

This paper compares the ranking of 12 state-of-the art saliency models using 12 similarity metrics and shows that some of the metrics are strongly correlated leading to a redundancy in the performance metrics reported in the available benchmarks.

Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model

This paper presents a novel model which can predict accurate saliency maps by incorporating neural attentive mechanisms, and shows, through an extensive evaluation, that the proposed architecture outperforms the current state-of-the-art on public saliency prediction datasets.

Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images

  • E. VigM. DorrD. Cox
  • Computer Science
    2014 IEEE Conference on Computer Vision and Pattern Recognition
  • 2014
This work identifies those instances of a richly-parameterized bio-inspired model family (hierarchical neuromorphic networks) that successfully predict image saliency and uses automated hyperparameter optimization to efficiently guide the search.

Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study

This study allows one to assess the state-of-the-art visual saliency modeling, helps to organizing this rapidly growing field, and sets a unified comparison framework for gauging future efforts, similar to the PASCAL VOC challenge in the object recognition and detection domains.

OpenSalicon: An Open Source Implementation of the Salicon Saliency Model

This technical report presents a publicly downloadable implementation of the SALICON saliency model, and provides a pre-trained model and code for those users who only need to generate saliency maps for images without training their own model.

Saliency Detection: A Boolean Map Approach

A novel Boolean Map based Saliency model, based on a Gestalt principle of figure-ground segregation, that consistently achieves state-of-the-art performance compared with ten leading methods on five eye tracking datasets.

DeepGaze II: Reading fixations from deep features trained on object recognition

The model uses the features from the VGG-19 deep neural network trained to identify objects in images for saliency prediction with no additional fine-tuning and achieves top performance in area under the curve metrics on the MIT300 hold-out benchmark.

Saliency, attention, and visual search: an information theoretic approach.

It is demonstrated that a variety of visual search behaviors appear as emergent properties of the model and therefore basic principles of coding and information transmission are demonstrated.