Visual speech improves the intelligibility of time-expanded auditory speech

@article{Tanaka2009VisualSI,
  title={Visual speech improves the intelligibility of time-expanded auditory speech},
  author={Akihiro Tanaka and Shuichi Sakamoto and Komi Tsumura and Y{\^o}iti Suzuki},
  journal={NeuroReport},
  year={2009},
  volume={20},
  pages={473-477}
}
This study investigated the effects of intermodal timing differences and speed differences on word intelligibility of auditory–visual speech. Words were presented under visual-only, auditory-only, and auditory–visual conditions. Two types of auditory–visual conditions were used: asynchronous and expansion conditions. In the asynchronous conditions, the audio lag was 0–400 ms. In the expansion conditions, the auditory signal was time expanded (0–400 ms), whereas the visual signal was kept at the… 
View on Wolters Kluwer
ais.riec.tohoku.ac.jp
10 Citations
Background Citations
4

10 Citations

Citation Type

Citation Type

Aging effect on audio-visual speech asynchrony perception: comparison of time-expanded speech and a moving image of a talker2s face
In this study, we measured detection and tolerance thresholds of auditory-visual asynchrony between time-expanded speech and a moving image of the talker’s face for older adults. During experiments,…
Effects of speech-rate conversion on asynchrony perception of audio-visual speech
TLDR
Examination of detection thresholds of auditory-visual asynchrony between time-expanded speech and moving image of the talker’s face by using words that have different numbers of mora but the same duration suggests that word length, not the number of MORa, affects the detection thresholds between auditory and visual stimuli.
The Auditory-Visual Speech Benefit on Working Memory in Older Adults with Hearing Impairment
TLDR
Support is provided for the theory of an integrated perceptual-cognitive system and the presence of visual speech cues may have helped the PH group to counteract the demanding auditory processing, to the level that no group differences were evident during the AV modality despite lower performance during the A-only condition.
Recalibration of audiovisual simultaneity in speech
TLDR
This work investigates whether the temporal recalibration occurs for audio-visual speech using an off-line adaptation method and suggests different properties of temporal recalibrration in speech.
Binding and unbinding the auditory and visual streams in the McGurk effect.
TLDR
If a given McGurK stimulus is preceded by an incoherent audiovisual context, the amount of McGurk effect is largely reduced and the data are interpreted in the framework of a two-stage "binding and fusion" model for audiovISual speech perception.
Sources and Correlates of Performance Enhancement in Audiovisual Speech Perception
TLDR
The author reveals that the McGurk Illusion is Negatively Correlated with Performance in an Audiovisual Speech-in-Noise Task and the author’s motivation to write this book was motivated by a desire to address this problem.
Perception of intersensory synchrony: A tutorial review
TLDR
Four mechanisms of how intersensory lags might be dealt with are identified: by ignoring lags up to some point (a wide window of temporal integration), by compensating for predictable variability, by adjusting the point of perceived synchrony on the longer term, and by shifting one stream directly toward the other.
Temporal Recalibration in Audio-Visual Speech Integration Using a Simultaneity Judgment Task and the McGurk Identification Task
Temporal Recalibration in Audio-Visual Speech Integration Using a Simultaneity Judgment Task and the McGurk Identification Task Kaori ASAKAWA 1 (kaori@ais.riec.tohoku.ac.jp) Division of Psychology,…
Perception of Synchrony between the Senses
Effects of pause duration and speech rate on sentence intelligibility in younger and older adult listeners

References

SHOWING 1-10 OF 23 REFERENCES
Effect of speed difference between time-expanded speech and talker2s moving image on word or sentence intelligibility
TLDR
Effects of asynchronicity between a speech signal and a talker’s moving image induced by time-expansion of the speech signal are investigated to suggest that the talker's moving image is effective to enhance speech intelligibility if the lag between thespeech signal and the talkers’moving image is less than or equal to 200 ms.
Perception of asynchronous and conflicting visual and auditory speech.
TLDR
The fuzzy logical model of perception (FLMP), which accurately describes integration, was used to measure the degree to which integration of audible and visible speech occurred and provide information about the temporal window of integration and its apparent dependence on the range of speech events in the test.
Speech intelligibility derived from asynchronous processing of auditory-visual information
TLDR
There is a marked asymmetry in the integration of audio and visual information that has important implications for sensorybased models of auditory-visual speech processing and in some instances may actually improve.
Temporal constraints on the McGurk effect
TLDR
The results suggest that perceivers may be sensitive to the concordance of the time-varying aspects of speech but they do not require temporal coincidence of that information.
Auditory-visual speech perception and synchrony detection for speech and nonspeech signals.
  • B. Conrey, D. Pisoni
  • Psychology, Physics
    The Journal of the Acoustical Society of America
  • 2006
TLDR
Variability in AV synchrony detection for both speech and nonspeech signals was found to be related to variability in measures of auditory-only (A-only) and AV speech perception, suggesting that temporal processing for bothspeech and nonsPEech signals must be taken into account in explaining variability in A-only and multisensory speech perception.
Visual contribution to speech intelligibility in noise
Oral speech intelligibility tests were conducted with, and without, supplementary visual observation of the speaker's facial and lip movements. The difference between these two conditions was…
Auditory influences on visual temporal rate perception.
  • G. Recanzone
  • Psychology, Biology
    Journal of neurophysiology
  • 2003
TLDR
The results show that the auditory system can strongly influence visual perception and are consistent with the idea that bimodal sensory conflicts are dominated by the sensory system with the greater acuity for the stimulus parameter being discriminated.
Temporal window of integration in auditory-visual speech perception
A procedure for measuring auditory and audio-visual speech-reception thresholds for sentences in noise: rationale, evaluation, and recommendations for use.
The strategy for measuring speech-reception thresholds for sentences in noise advocated by Plomp and Mimpen (Audiology, 18, 43-52, 1979) was modified to create a reliable test for measuring the…
Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments.
TLDR
It is contended that the multisensory speech system is maximally tuned for SNRs between extremes, where the system relies on either the visual (speech-reading) or the auditory modality alone, forming a window of maximal integration at intermediate SNR levels.
...
1
2
3
...