In this paper, we present an overview of research in our laboratories on Multimodal Human Computer Interfaces. The goal for such interfaces is to free human computer interaction from the limitations and acceptance barriers due to rigid operating commands and keyboards as only/main I/O-device. Instead we move to involve all available human communication… (More)
Modern user interfaces can take advantage of multiple input modalities such as speech, gestures, handwriting... to increase robustness and flexibility. The construction of such multimodal interfaces would be greatly facilitated by a unified framework that provides methods to characterize and interpret multimodal inputs. In this paper we describe a semantic… (More)
While human-to-human communication takes advantage of an abundance of information and cues, human-computer interaction is limited to only a few input modalities (usually only keyboard and mouse) and provides little flexibility as to choice of communication modality. In this paper, we present an overview of a family of research projects we are undertaking at… (More)
Multi-modal interfaces can achieve more natural and effective human-computer interaction by integrating a variety of signal% or modalities, by which humans usually convey information. The integration of multiple input modalities permits greater expressiveness tim complementary information sources, and greater reliability due to redundancies across… (More)
The Time Delay Neural Network (TDNN) is one of the neural network architectures that give excellent performance in tasks involving classification of temporal signals, such as phoneme classification, on-line gesture and handwriting recognition, and many others. One particular problem that occurs in on-line recognition tasks is how to deal with input patterns… (More)
In this paper, we present an adaptive multimodal interface for wireless computing applications. The system optimizes its performance by dynamically selecting its network service based on cost and performance requirements. We describe a prototype system and its application to demonstrate the proposed concept.