An improved automatic lipreading system to enhance speech recognition

@inproceedings{Petajan1988AnIA,
  title={An improved automatic lipreading system to enhance speech recognition},
  author={Eric Petajan and Birgit Bischoff and David Bodoff and N. Michael Brooke},
  booktitle={CHI '88},
  year={1988}
}
Current acoustic speech recognition technology performs well with very small vocabularies in noise or with large vocabularies in very low noise. Accurate acoustic speech recognition in noise with vocabularies over 100 words has yet to be achieved. Humans frequently lipread the visible facial speech articulations to enhance speech recognition, especially when the acoustic signal is degraded by noise or hearing impairment. Automatic lipreading has been found to improve significantly acoustic… Expand
Computer lipreading for improved accuracy in automatic speech recognition
TLDR
An audiovisual system known as Lipreading to Enhance Automatic Perception of Speech (LEAPS) is described, in which the lipreading system is used in conjunction with an audio ASR system in order to improve the accuracy of the latter, especially under degraded acoustical conditions. Expand
1 Using Surface-Learning to improve Speech Recognition with Lipreading
We explore multimodal recognition by combining visual lipreading with acoustic speech recognition. We show that combining the visual and acoustic clues of speech improves the recog­ nitionExpand
Combining Visual and Acoustic Speech Signals with a Neural Network Improves Intelligibility
TLDR
An alternative approach to fusing the visual and acoustic speech information by training feedforward neural networks to map the visual signal onto the corresponding short-term spectral amplitude envelope (STSAE) of the acoustic signal is demonstrated. Expand
Neural network lipreading system for improved speech recognition
A modified time-delay neural network (TDNN) has been designed to perform both automatic lipreading (speech reading) in conjunction with acoustic speech recognition in order to improve recognitionExpand
Improving connected letter recognition by lipreading
The authors show how recognition performance in automated speech perception can be significantly improved by additional lipreading, so called speech-reading. They show this on an extension of aExpand
Automatic speech recognition system using acoustic and visual signals
  • M. Hennecke, K. Prasad, D. Stork
  • Computer Science
  • Conference Record of The Twenty-Ninth Asilomar Conference on Signals, Systems and Computers
  • 1995
TLDR
This paper presents a complete speech-reading system, which is able to record an utterance using a standard color video camera, preprocess both the audio and video signal, and perform speech recognition. Expand
Features for Audio-Visual Speech Recognition
TLDR
Five new lipreading techniques are evaluated using a hidden Markov model based visual-only recognition task and compared with an enhanced implementation of a previous lip contour tracker, finding the addition of visual information to automatic speech recognition is found to improve accuracy and is most pronounced in acoustically noisy conditions. Expand
Automatic Lipreading Research: Historic Overview and Current Work
TLDR
The history of automatic lipreading research is outlined, the authors current efforts are described and the use of visual speech (lipreading or, more properly, speechreading) information has been shown to improve the performance of acoustic ASR systems especially in noise. Expand
A hybrid approach to bimodal speech recognition
TLDR
It is shown that combining visual and acoustic speech information improves the recognition performance significantly, especially in noisy environments, with a hybrid speech recognition architecture. Expand
Audiovisual speech/speaker recognition, application to Arabic language
  • F. Chelali, A. Djeradi
  • Computer Science
  • 2011 International Conference on Multimedia Computing and Systems
  • 2011
TLDR
A speaker identification system where lip information is fused with corresponding speech information from each speaker using a multilayer perceptron classifier and the energy, the zero cross ratio (ZCR) and the pitch are used as features for the audio modality. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 17 REFERENCES
Automatic lipreading to enhance speech recognition (speech reading)
TLDR
An automatic lipreading system which has been developed and the combination of the acoustic and visual recognition candidates is shown to yield a final recognition accuracy which greatly exceeds the acoustic recognition accuracy alone. Expand
The Effects on Computer Recognition of Speech When Speaking through Protective Masks
Abstract : The purpose of this experiment was to investigate the effects on computer recognition of speech when speaking through aviator protective masks as compared to the standard boom-mountedExpand
Dynamic programming algorithm optimization for spoken word recognition
TLDR
This paper reports on an optimum dynamic progxamming (DP) based time-normalization algorithm for spoken word recognition, in which the warping function slope is restricted so as to improve discrimination between words in different categories. Expand
Vector quantization: A pattern-matching technique for speech coding
TLDR
Recent results obtained in waveform coding of speech with vector quantization are reviewed, with Vector quantization appearing to be a suitable coding technique which caters to this dual requirement of effective speech coding. Expand
Coding Of Two-Tone Images
We give a brief overview of efficient coding methods for two-tone images, especially: white block skipping and runlength coding.
A Comparative Evaluation of Speech Recognizers
  • Proc. Speech Tech '87
  • 1987
A Comparative Evaluation of Speech Recognizers,
  • Proc. Speech Tech
  • 1987
A Natural Speech Recognition System,
  • Proc. Speech Tech
  • 1987
An Architecture for High Speed Contour and Region Coding of Threshold Images
  • Proc . IEEE International Conference on Computer Design : VLSI in Computers
  • 1986
An Investigation of Visible Lip Information to be Used in Automated Speech Recognition
  • An Investigation of Visible Lip Information to be Used in Automated Speech Recognition
  • 1986
...
1
2
...