Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes

@inproceedings{Vogler1999TowardSI,
  title={Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes},
  author={Christian Vogler and Dimitris N. Metaxas},
  booktitle={Gesture Workshop},
  year={1999}
}
In this paper we present a novel approach to continuous, whole-sentence ASL recognition that uses phonemes instead of whole signs as the basic units. Our approach is based on a sequential phonological model of ASL. According to this model the ASL signs can be broken into movements and holds, which are both considered phonemes.This model does away with the distinction between whole signs and epenthesis movements that we made in previous work [17]. Instead, epenthesis movements are just like the… 
View via Publisher
cs.bu.edu
156 Citations
Highly Influential Citations
7
Background Citations
48
Methods Citations
30
Results Citations
4

156 Citations

Citation Type

Citation Type

American sign language recognition: reducing the complexity of the task with phoneme-based modeling and parallel hidden markov models
TLDR
This thesis presents a framework for recognizing American Sign Language (ASL) from 3D data, and develops a PaHMM recognition algorithm specifically geared toward the properties of sign languages, and poses parallel hidden Markov models (PaHMMs) as an extension to conventional hiddenMarkov models.
A Framework for Recognizing the Simultaneous Aspects of American Sign Language
TLDR
This paper presents a novel framework to ASL recognition that aspires to being a solution to the scalability problems, based on breaking down the signs into their phonemes and modeling them with parallel hidden Markov models.
An approach based on phonemes to large vocabulary Chinese sign language recognition
  • Chunli Wang, S. Shan, Wen Gao
  • Computer Science
    Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition
  • 2002
TLDR
This work presents an approach to large vocabulary, continuous Chinese sign language (CSL) recognition that uses phonemes instead of whole signs as the basic units, and facilitates the CSL recognition when the finger-alphabet is blended with gestures.
Handshapes and movements: Multiple-channel ASL recognition
TLDR
This paper loosely follow the Movement-Hold model to devise a breakdown of the signs into their constituent phonemes, which provide the fundamental building blocks, and shows how to integrate the handshape into this breakdown, and discusses what handshape representation works best.
A Comparison Between Etymon- and Word-Based Chinese Sign Language Recognition Systems
TLDR
Two approaches to large vocabulary Chinese Sign Languagerecognition are discussed and etymon- and word- based recognition systems are introduced, which are designed to recognize 2439 etyma and 5100 signs.
Parallel hidden Markov models for American sign language recognition
TLDR
A novel approach to ASL recognition that aspires to being a solution to the scalability problems, based on parallel HMMs (PaHMMs), which model the parallel processes independently and can be trained independently, and do not require consideration of the different combinations at training time.
Signer-independent fingerspelling recognition with deep neural network adaptation
TLDR
Using DNN adaptation, it is found that it is possible to bridge a large part of the gap between signer-dependent and signing-independent performance.
Sign language phoneme transcription with PCA-based representation
  • W. Kong, S. Ranganath
  • Computer Science
    2007 6th International Conference on Information, Communications & Signal Processing
  • 2007
TLDR
A semi-automatic segmentation algorithm is applied which detects minimal velocity and maximal change of directional angle to segment the hand motion trajectory of signed sentences and feature descriptors based on principal component analysis (PCA) are extracted to represent the segments efficiently.
Extension of hidden markov model for recognizing large vocabulary of sign language
TLDR
A novel approach based on HMM is presented to reduce the recognition complexity of French Sign Language (FSL), one of the most accomplished forms of gestural communication.
Sign Language Phoneme Transcription with Rule-based Hand Trajectory Segmentation
TLDR
A rule-based segmentation algorithm is applied to segment the hand motion trajectories of signed sentences and feature descriptors based on principal component analysis are extracted to represent the segments efficiently.
...
...

References

SHOWING 1-10 OF 32 REFERENCES
Parallel hidden Markov models for American sign language recognition
TLDR
A novel approach to ASL recognition that aspires to being a solution to the scalability problems, based on parallel HMMs (PaHMMs), which model the parallel processes independently and can be trained independently, and do not require consideration of the different combinations at training time.
American Sign Language: The Phonological Base
This paper has the ambitious goal of outlining the phonological structures and processes we have analyzed in American Sign Language (ASL). In order to do this we have divided the paper into five…
Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods
TLDR
An approach to continuous American sign language (ASL) recognition, which uses as input 3D data of arm motions and training context-dependent HMMs and is inspired by speech recognition systems is presented.
Visual Recognition of American Sign Language Using Hidden Markov Models.
TLDR
Using hidden Markov models (HMM's), an unobstrusive single view camera system is developed that can recognize hand gestures, namely, a subset of American Sign Language (ASL), achieving high recognition rates for full sentence ASL using only visual cues.
Isolated ASL sign recognition system for deaf persons
TLDR
The design and evaluation of a two-stage neural network which can recognize isolated ASL signs is given and the network could not only recognize the signs with 84% accuracy, but also expand its learned vocabulary through relabeling.
ASL recognition based on a coupling between HMMs and 3D motion analysis
TLDR
It is demonstrated that context-dependent modeling and the coupling of vision methods and HMMs improve the accuracy of continuous ASL recognition.
A real-time continuous gesture recognition system for sign language
TLDR
A large vocabulary sign language interpreter is presented with real-time continuous gesture recognition of sign language using a data glove using hidden Markov models for 51 fundamental postures, 6 orientations, and 8 motion primitives.
Coupled hidden Markov models for complex action recognition
  • M. Brand, N. Oliver, A. Pentland
  • Computer Science
    Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • 1997
We present algorithms for coupling and training hidden Markov models (HMMs) to model interacting processes, and demonstrate their superiority to conventional HMMs in a vision task classifying…
Recognition of space-time hand-gestures using hidden Markov model
TLDR
A hidden Markov(HMM)-based method for recognizing the space-time hand movement pattern and the three major attributes, which are regarded as the primary attribute of the hand gesture, are considered.
A MULTI-STAGE APPROACH TO FINGERSPELLING AND GESTURE RECOGNITION
TLDR
The problem of finding a hierarchy of classifiers, optimal with respect to recognition accuracy, given a labeled corpus of signing samples is addressed, and the best algorithm attains significant increase in accuracy and exemplifies crucial distinction between finding an optimal hierarchy and clustering.
...
...