Sequential Belief-Based Fusion of Manual and Non-manual Information for Recognizing Isolated Signs

@inproceedings{Aran2007SequentialBF,
  title={Sequential Belief-Based Fusion of Manual and Non-manual Information for Recognizing Isolated Signs},
  author={Oya Aran and Thomas Burger and Alice Caplier and Lale Akarun},
  booktitle={Gesture Workshop},
  year={2007}
}
This work aims to recognize signs which have both manual and non-manual components by providing a sequential belief-based fusion mechanism. We propose a methodology based on belief functions for fusing extracted manual and non-manual features in a sequential two-step approach. The belief functions based on the likelihoods of the hidden Markov models are used to decide whether there is an uncertainty in the decision of the first step and also to identify the uncertainty clusters. Then we proceed… 
Two k-additive generalizations of the pignistic transform for imprecise decision making
TLDR
A framework for imprecise decision derived from the TBM is proposed, in which belief functions are mapped to k-additive belief functions (i.e., belief functions whose focal elements have maximal cardinality equal to k) rather than Bayesian ones.
Towards Automatic Annotation of Sign Language Dictionary Corpora
TLDR
A new method for automatic parameterization of these video files and categorization of the signs from extracted information is designed and incorporates advanced image processing for detection and tracking of hands and head of signing character in the input image sequences.
Visions of a generalized probability theory
TLDR
It is argued that the fruitful interaction of computer vision and belief calculus is capable of stimulating significant advances in both fields, and some initial steps towards a generalization of the notion of total probability to belief functions are taken.
Correlation analysis of facial features and sign gestures
TLDR
A simple but robust method is presented for the hand tracking to obtain a 2D trajectory representing a portion of the manual component and the most correlated signals are the vertical movement of head and hands.
Recognizing eyebrow and periodic head gestures using CRFs for non-manual grammatical marker detection in ASL
  • Jingjing Liu, Bo Liu, C. Neidle
  • Computer Science
    2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)
  • 2013
TLDR
A 2-level CRF method for recognition of the components of eyebrow and periodic head gestures, differentiating the linguistically significant domain (core) from transitional movements (which the authors refer to as the onset and offset), and uses a robust face tracker and 3D warping to extract and combine the geometric and appearance features.
Recognition of Nonmanual Markers in American Sign Language (ASL) Using Non-Parametric Adaptive 2D-3D Face Tracking
TLDR
The proposed new framework makes it possible to detect grammatically significant nonmanual expressions from continuous signing and to differentiate successfully among linguistically significant expressions that involve subtle differences in appearance.
Machine learning based sign language recognition: a review and its research frontier
  • R. Elakkiya
  • Computer Science
    J. Ambient Intell. Humaniz. Comput.
  • 2021
TLDR
The impact of machine learning in the state of the art literature on sign language recognition and classification is investigated and the potential gaps that machine learning approaches need to address for the real-time signlanguage recognition are discussed.
Classification of extreme facial events in sign language videos
TLDR
ESC is applied to detect visual events on SL videos, including both American and Greek corpora, yielding promising qualitative and quantitative results and shows the potential of ESC for assistive annotation tools and a link of the detections with indicative higher-level linguistic events.
...
...

References

SHOWING 1-10 OF 19 REFERENCES
Modeling Hesitation and Conflict: A Belief-Based Approach for Multi-class Problems
TLDR
This paper presents a method which partially prevents two losses of information by applying belief theories (BTs) to SVM fusion, while keeping the efficient aspect of the classical methods.
A Mathematical Theory of Evidence
  • G. Shafer
  • Mathematics
    A Mathematical Theory of Evidence
  • 2020
TLDR
This book develops an alternative to the additive set functions and the rule of conditioning of the Bayesian theory: set functions that need only be what Choquet called "monotone of order of infinity." and Dempster's rule for combining such set functions.
Automatic detection of relevant head gestures in American Sign Language communication
  • U. M. Erdem, S. Sclaroff
  • Computer Science
    Object recognition supported by user interaction for service robots
  • 2002
An automated system for detection of head movements is described The goal is to label relevant head gestures in video of American Sign Language (ASL) communication. In the system, a 3D head tracker
Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods
TLDR
An approach to continuous American sign language (ASL) recognition, which uses as input 3D data of arm motions and training context-dependent HMMs and is inspired by speech recognition systems is presented.
Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning
TLDR
Data acquisition, feature extraction and classification methods employed for the analysis of sign language gestures are examined and the overall progress toward a true test of sign recognition systems--dealing with natural signing by native signers is discussed.
Parallel hidden Markov models for American sign language recognition
TLDR
A novel approach to ASL recognition that aspires to being a solution to the scalability problems, based on parallel HMMs (PaHMMs), which model the parallel processes independently and can be trained independently, and do not require consideration of the different combinations at training time.
Robust Real-Time Face Detection
TLDR
A new image representation called the “Integral Image” is introduced which allows the features used by the detector to be computed very quickly and a method for combining classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising face-like regions.
Handshapes and Movements: Multiple-Channel American Sign Language Recognition
TLDR
A framework for recognizing American Sign Language (ASL) and the main challenges in developing scalable recognition systems are to devise the basic building blocks from which to build up the signs, and to handle simultaneous events.
Head nods analysis: interpretation of non verbal communication gestures
TLDR
A real time frequency method to detect 2D rigid rotations of pan or tilt of a moving head and interpret head nods involved in the non verbal communication process in the same way as human being is proposed.
Hand modeling, analysis and recognition
TLDR
3-D hand models, various articulated motion analysis methods, and gesture recognition techniques employed in current research are studied, some of which are shown as examples.
...
...