Contributions of oral and extraoral facial movement to visual and audiovisual speech perception.

@article{Thomas2004ContributionsOO,
  title={Contributions of oral and extraoral facial movement to visual and audiovisual speech perception.},
  author={Sharon M. Thomas and Timothy R Jordan},
  journal={Journal of experimental psychology. Human perception and performance},
  year={2004},
  volume={30 5},
  pages={
          873-88
        }
}
Seeing a talker's face influences auditory speech recognition, but the visible input essential for this influence has yet to be established. Using a new seamless editing technique, the authors examined effects of restricting visible movement to oral or extraoral areas of a talking face. In Experiment 1, visual speech identification and visual influences on identifying auditory speech were compared across displays in which the whole face moved, the oral area moved, or the extraoral area moved… 
View on PubMed
doi.org
80 Citations
Highly Influential Citations
5
Background Citations
33
Methods Citations
4
Results Citations
4

Figures from this paper

80 Citations

Citation Type

Citation Type

When half a face is as good as a whole: Effects of simple substantial occlusion on visual and audiovisual speech perception
TLDR
It is shown that visual and audiovisual speech perception also functioned well with other simple substantial occlusions (horizontal and diagonal), and displays in which entire upper facial areas were occluded produced performance levels equal to those obtained with unoccluded displays.
Differential Gaze Patterns on Eyes and Mouth During Audiovisual Speech Segmentation
TLDR
Eye-gaze patterns changed across familiarization as subjects learned the word boundaries, showing decreased attention to the mouth in later blocks while attention on other facial features remained consistent, highlighting the importance of the visual component of speech processing and suggesting that the mouth may play a critical role in visual speech segmentation.
Can you McGurk yourself? Self-face and self-voice in audiovisual speech
TLDR
In the present experiment, the relative contributions of facial and vocal inputs to speech perception were evaluated with an audiovisual illusion and indicate that auditory self-speech conveys a processing advantage, whereas visual self- speech does not.
Audiovisual speech perception: A new approach and implications for clinical populations
TLDR
This selected overview of audiovisual (AV) speech perception examines the influence of visible articulatory information on what is heard and indicates potential loci, timing, and facilitatory effects of AV over auditory only speech.
Cognitive resources in audiovisual speech perception
Most events that we encounter in everyday life provide our different senses with correlated information, and audiovisual speech perception is a familiar instance of multisensory integration. Several…
Effects of mouth-only and whole-face displays on audio-visual speech perception in noise: is the vision of a talker's full face truly the most efficient solution?
TLDR
The results showed that the facilitative effect of visual information on speech perception depended upon both auditory input degradation level and the visual presentation format, while eye-movement behavior was only affected by the visual input format.
Audiovisual speech integration and lipreading in autism.
  • E. Smith, L. Bennetto
  • Psychology
    Journal of child psychology and psychiatry, and allied disciplines
  • 2007
TLDR
Combined deficits in audiovisual speech integration and lipreading in individuals with autism are likely to contribute to ongoing difficulties in speech comprehension, and may also be related to delays in early language development.
Timing in audiovisual speech perception: A mini review and new psychophysical data
TLDR
A classification procedure is introduced that tracks perceptually relevant visual speech information in time without requiring artificial manipulation of cross-modal alignment or stimulus duration and results show temporally-leading visual information did influence auditory perception.
The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception
Coordination of eyebrow movement with speech acoustics and head movement
TLDR
There was a correlation between eyebrow movements and speech acoustics, though there was notable variation within and across participants, and the implications for theories of gestural control in speech production are discussed.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 99 REFERENCES
Effects of horizontal viewing angle on visual and audiovisual speech recognition.
  • T. R. Jordan, S. Thomas
  • Psychology
    Journal of experimental psychology. Human perception and performance
  • 2001
TLDR
It is indicated that audiovisual speech recognition withstands substantial changes in horizontal viewing angle, but explicit identification of visual speech is less robust.
Seeing and hearing rotated faces: influences of facial orientation on visual and audiovisual speech recognition.
  • T. R. Jordan, K. Bevan
  • Psychology
    Journal of experimental psychology. Human perception and performance
  • 1997
TLDR
Three experiments are reported in which the effects of facial orientation on visual speech processing were examined using a talking face presented at 8 orientations through 360 degrees, finding that facial orientation did affect the accuracy of auditory speech report with incongruent audiovisual speech stimuli.
Determining the influence of Gaussian blurring on inversion effects with talking faces
TLDR
This work examines the contributions of visually coarse and fine facial movement information to inversion effects in the perception of visual and audiovisual speech and disrupts perception of fine facial detail by decreasing spatial frequency (blurring) and of coarse configural information by facial inversion.
Visual and audiovisual speech perception with color and gray-scale facial images
TLDR
It is indicated that luminance, rather than color, underlies visual and audiovisual speech perception and that this information is more than the kinematic information provided by point-light faces.
Effects of Distance on Visual and Audiovisual Speech Recognition
Face-to-face conversations in every day life are conducted over a range of distances. However, previous research provides only limited indications of the effects of distance on visual and audiovisual…
Effects of horizontal viewing angle on visual and audiovisual speech perception
Discovering how humans extract visual speech from faces viewed at different horizontal angles remains an outstanding goal for understanding brain function and for implementing human speech perception…
Laterality in visual speech perception.
TLDR
The results suggest that the left-hemisphere advantage is based on the processing of dynamic visual information rather than on the extraction of linguistic significance from facial movements.
The Dynamics of Audiovisual Behavior in Speech
TLDR
This work extends the analysis to full facial motion under the assumption that the process of producing speech acoustics generates linguistically salient visual information, which is distributed over large portions of the face, which could become a useful experimental tool, providing synthetic audiovisual stimuli with realistic control parameters.
Hearing by eye 2 : advances in the psychology of speechreading and auditory-visual speech
Part 1 Audiovisual speech processing - implications for theories of speech perception: the use of auditory and visual information during phonetic processing - implications for theories of speech…
Face and mouth inversion effects on visual and audiovisual speech perception.
TLDR
Results suggest that upright facial context can play a role in visual speech perception, and suggest that an upright face with an inverted mouth and an inverted face containing an upright mouth look relatively normal.
...
1
2
3
4
5
...