The role of visual experience in the emergence of cross-modal correspondences

@article{HamiltonFletcher2018TheRO,
  title={The role of visual experience in the emergence of cross-modal correspondences},
  author={Giles Hamilton-Fletcher and K. Pisanski and D. Reby and M. Stefańczyk and J. Ward and A. Sorokowska},
  journal={Cognition},
  year={2018},
  volume={175},
  pages={114-121}
}
Cross-modal correspondences describe the widespread tendency for attributes in one sensory modality to be consistently matched to those in another modality. For example, high pitched sounds tend to be matched to spiky shapes, small sizes, and high elevations. However, the extent to which these correspondences depend on sensory experience (e.g. regularities in the perceived environment) remains controversial. Two recent studies involving blind participants have argued that visual experience is… Expand
Temperature-Based Crossmodal Correspondences: Causes and Consequences.
  • C. Spence
  • Psychology, Medicine
  • Multisensory research
  • 2020
TLDR
Gaining a better understanding of temperature-based crossmodal correspondences may one day help in the design of more intuitive sensory-substitution devices, and support the delivery of immersive virtual and augmented reality experiences. Expand
The feeling of "kiki": Comparing developmental changes in sound-shape correspondence for audio-visual and audio-tactile stimuli.
TLDR
The finding suggests that AT sound- shape correspondence is weak early in development relative to AV sound-shape correspondence, paralleling previous findings on the development of other types of multisensory associations. Expand
Sound symbolism in sighted and blind. The role of vision and orthography in sound-shape correspondences
TLDR
The hypothesis that visual experience is necessary for the emergence of non-arbitrary sound-shape correspondences (SSC) is supported by empirical evidence showing reduced SSC in visually impaired people, and differences between sighted and visually impairment people may be due the indirect influence of orthographic letter shape. Expand
Heaviness-brightness correspondence and stimulus-response compatibility
TLDR
Heaviness, alongside size, appears to contribute to cross-sensory correspondences in its own right and in a manner confirming the far-reaching influence of correspondences, extending here to the fluency with which people communicate simple ideas by manipulating a hand-held object. Expand
A Protracted Sensitive Period Regulates the Development of Cross-Modal Sound–Shape Associations in Humans
TLDR
Evidence is provided for a protracted sensitive period during which aberrant vision prevents SSA acquisition and the finding of a systematic SSA in the LB group demonstrates that representations acquired during the sensitive period are resilient to loss despite dramatically changed experience. Expand
People With High Autistic Traits Show Fewer Consensual Crossmodal Correspondences Between Visual Features and Tastes
  • Na Chen, Katsumi Watanabe, Makoto Wada
  • Frontiers in Psychology
  • 2021
Crossmodal correspondences between visual features (e.g., color/shape) and tastes have been extensively documented in recent years. Visual colors and shapes have been shown to consensually match toExpand
How Much Spatial Information Is Lost in the Sensory Substitution Process? Comparing Visual, Tactile, and Auditory Approaches
TLDR
It is concluded that this process is best benchmarked against performance with SSDs that return to their primary modality (e.g., visuospatial into visual) and ways of closing this "modality gap" found in SSDs are discussed. Expand
Brain networks underlying the processing of sound symbolism related to softness perception
TLDR
Results indicate that the insula and medial superior frontal gyrus play a role in processing sound symbolic information and relating it to the tactile softness information. Expand
Vocal communication of magnitude across language, age, and auditory experience.
Like many other vocalizing vertebrates, humans convey information about their body size through the sound of their voice. Vocalizations of larger animals are typically longer in duration, louder inExpand
The link between auditory salience and emotion intensity
  • A. Anikin
  • Medicine, Psychology
  • Cognition & emotion
  • 2020
TLDR
The acoustic code in nonverbal communication is aligned with sensory biases, offering a general explanation for some non-arbitrary properties of human and animal high-arousal vocalizations. Expand
...
1
2
...

References

SHOWING 1-10 OF 52 REFERENCES
Audio-Visual, Visuo-Tactile and Audio-Tactile Correspondences in Preschoolers.
TLDR
The results suggest that pitch-height correspondences for audio-visual and audio-tactile combinations may still be weak in preschool children, and speculate that this could be due to immature linguistic and auditory cues that are still developing at age five. Expand
Summation of visual attributes in auditory–visual crossmodal correspondences
TLDR
This work investigated how the visual dimensions of luminance, saturation, size, and vertical position can influence decisions when matching particular visual stimuli with high-pitched or low-pitches auditory stimuli. Expand
Preverbal Infants’ Sensitivity to Synaesthetic Cross-Modality Correspondences
TLDR
3- to 4-month-old preverbal infants’ sensitivity to the correspondences linking auditory pitch to visuospatial height and visual sharpness is assessed, giving the strongest indication to date that synaesthetic cross-modality correspondences are an unlearned aspect of perception. Expand
Crossmodal correspondences: A tutorial review
  • C. Spence
  • Psychology, Medicine
  • Attention, perception & psychophysics
  • 2011
TLDR
The literature reviewed here supports the view thatCrossmodal correspondences need to be considered alongside semantic and spatiotemporal congruency, among the key constraints that help the authors' brains solve the crossmodal binding problem. Expand
Newborns are sensitive to the correspondence between auditory pitch and visuospatial elevation
TLDR
Newborns' looking behavior was sensitive to this congruence, providing the strongest indication to date that cross-sensory correspondences can be in place at birth. Expand
Natural cross-modal mappings between visual and auditory features.
TLDR
In a series of speeded classification tasks, spontaneous mappings between the auditory feature of pitch and the visual features of vertical location, size, and spatial frequency are found but not contrast. Expand
Differentiated audio-tactile correspondences in sighted and blind individuals.
TLDR
Whether the crossmodal correspondence robustly documented between auditory pitch and visual elevation has analogues in the audio-tactile domain and the origin of the widely discussed association between pitch and vertical elevation are investigated are investigated. Expand
Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans
TLDR
An inherent tendency to pair high pitch with high luminance hence evolved before the human lineage split from that of chimpanzees, and constitutes a basic feature of the primate sensory system. Expand
Crossmodal Correspondences: Standing Issues and Experimental Guidelines.
  • C. Parise
  • Psychology, Medicine
  • Multisensory research
  • 2016
TLDR
Questions about their definition, origins, their plasticity, and their underlying computational mechanisms are reviewed in the light of current research on sensory cue integration, where crossmodal correspondences can be conceptualized in terms of natural mappings across different sensory cues that are present in the environment and learnt by the sensory systems. Expand
Do small white balls squeak? Pitch-object correspondences in young children
TLDR
Children’s matching of pitch and size cannot be attributed to intensity matching or to learning, and these data support the hypothesis that some cross-modal correspondences are remnants of the neural mechanisms underlying neonatal perception. Expand
...
1
2
3
4
5
...