Meaning and Attentional Guidance in Scenes: A Review of the Meaning Map Approach

@article{Henderson2019MeaningAA,
  title={Meaning and Attentional Guidance in Scenes: A Review of the Meaning Map Approach},
  author={John M. Henderson and Taylor R. Hayes and Candace Elise Peacock and Gwendolyn L Rehrig},
  journal={Vision},
  year={2019},
  volume={3}
}
Perception of a complex visual scene requires that important regions be prioritized and attentionally selected for processing. What is the basis for this selection? Although much research has focused on image salience as an important factor guiding attention, relatively little work has focused on semantic salience. To address this imbalance, we have recently developed a new method for measuring, representing, and evaluating the role of meaning in scenes. In this method, the spatial distribution… 

Figures from this paper

Meaning guides attention during scene viewing, even when it is irrelevant
TLDR
The role of meaning is examined using two tasks in which meaning was irrelevant and salience was relevant: a brightness rating task and a brightness search task, which support theories in which scene semantics play a dominant role in attentional guidance in scenes.
Rapid Extraction of the Spatial Distribution of Physical Saliency and Semantic Informativeness from Natural Scenes in the Human Brain
TLDR
Findings are in line with previous evidence indicating that saliency is computed rapidly, while also indicating that information related to the spatial distribution of semantically informative scene elements is computed shortly thereafter, early enough to potentially exert an influence on eye movements.
Semantic object-scene inconsistencies affect eye movements, but not in the way predicted by contextualized meaning maps
TLDR
It is demonstrated that semantically inconsistent objects are experienced as less meaningful than their consistent counterparts, and that contextualized meaning maps do not capture prototypical influences of image meaning on gaze guidance.
Semantic object-scene inconsistencies affect eye movements, but not in the way predicted by contextualized meaning maps
Semantic information is important in eye-movement control. An important semantic influence on gaze guidance relates to object-scene relationships: objects that are semantically inconsistent with the
Meaning and expected surfaces combine to guide attention during visual search in scenes
TLDR
These findings suggest that the visual system prioritizes meaningful scene regions on target-related surfaces during visual search in scenes.
Looking for Semantic Similarity: What a Vector-Space Model of Semantics Can Tell Us About Attention in Real-World Scenes
TLDR
It is concluded that object semantics play a critical role in guiding attention through real-world scenes with a strong positive relationship between the semantic similarity of a scene region and viewers' focus of attention.
Meaning and Expected Surfaces Combine to Guide Attention During Visual Search in Scenes
How do spatial constraints and meaningful scene regions interact to control overt attention during visual search for objects in real-world scenes? To answer this question, we combined novel surface
Semantic object-scene inconsistencies affect eye movements, but not in the way predicted by contextualized meaning maps
Semantic information is important in eye movement control. An important semantic influence on gaze guidance relates to object-scene relationships: objects that are semantically inconsistent with the
When scenes speak louder than words: Verbal encoding does not mediate the relationship between scene meaning and visual attention.
TLDR
Seminal information in the scene steers the attentional ship, consistent with cognitive guidance theory, and verbal encoding processes do not appear to modulate the relationship between scene meaning and visual attention.
Salience-based object prioritization during active viewing of naturalistic scenes in young and older adults
TLDR
Reconciling the salience view with the object view, it is suggested that visual salience contributes to prioritization among objects, and the data point towards an increasing relevance of object-bound information with increasing age.
...
1
2
3
...

References

SHOWING 1-10 OF 62 REFERENCES
Meaning Guides Attention during Real-World Scene Description
TLDR
The results showed that both meaning and Salience predicted the spatial distribution of attention in these tasks, but that when the correlation between meaning and salience was statistically controlled, only meaning accounted for unique variance in attention.
Meaning-based guidance of attention in scenes as revealed by meaning maps
TLDR
It is proposed that meaning plays the dominant role in guiding human attention through scenes, and developed ‘meaning maps’ that represent the semantic richness of scene regions in a format that can be directly compared to image salience to demonstrate this.
Meaning guides attention during scene viewing, even when it is irrelevant
TLDR
The role of meaning is examined using two tasks in which meaning was irrelevant and salience was relevant: a brightness rating task and a brightness search task, which support theories in which scene semantics play a dominant role in attentional guidance in scenes.
Meaning guides attention in real-world scene images: Evidence from eye movements and meaning maps
TLDR
The results showed that both meaning and image salience predicted the duration-weighted distribution of attention, but that when the correlation between meaning and salience was statistically controlled, meaning accounted for unique variance in attention whereas salience did not.
How Does the Purpose of Inspection Influence the Potency of Visual Salience in Scene Perception?
TLDR
The results suggest that salience maps are not used to guide the eyes in these tasks, that cognitive override by task demands can be total, and that modelling top – down search is important but may not be easily accomplished within a salience-map framework.
Guidance of visual attention by semantic information in real-world scenes
TLDR
A review of experimental work on how different components of context, especially semantic information, affect attentional deployment is provided, highlighting work from the areas of object recognition, scene perception, and visual search.
Eye guidance in natural vision: reinterpreting salience.
TLDR
It is argued that there is a need to move away from this class of model and find the principles that govern gaze allocation in a broader range of settings, because the stimulus context is limited, and the dynamic, task-driven nature of vision is not represented.
Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search.
TLDR
An original approach of attentional guidance by global scene context is presented that combines bottom-up saliency, scene context, and top-down mechanisms at an early stage of visual processing and predicts the image regions likely to be fixated by human observers performing natural search tasks in real-world scenes.
Shifts in selective visual attention: towards the underlying neural circuitry.
TLDR
This study addresses the question of how simple networks of neuron-like elements can account for a variety of phenomena associated with this shift of selective visual attention and suggests a possible role for the extensive back-projection from the visual cortex to the LGN.
Computational modelling of visual attention
TLDR
Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment, providing a framework for a computational and neurobiological understanding of visual attention.
...
1
2
3
4
5
...