Fr ra am me ew wo or rk k a an nd d T To oo ol lk ki it t f fo or r t th he e C Co on ns st tr ru uc ct ti io on n o of f M Mu ul lt ti im mo od da al l L Le ea ar rn ni in ng g I In nt te er rf fa ac ce es s ABSTRACT Multimodal human-computer interaction, in which the computer accepts input from multiple channels or modalities, is more flexible, natural,… (More)
In this paper, we present an overview of research in our laboratories on Multimodal Human Computer Interfaces. The goal for such interfaces is to free human computer interaction from the limitations and acceptance barriers due to rigid operating commands and keyboards as only/main I/O-device. Instead we move to involve all available human communication… (More)
While human-to-human communication takes advantage of an abundance of information and cues, human-computer interaction is limited to only a few input modalities (usually only keyboard and mouse) and provides little flexibility as to choice of communication modality. In this paper, we present an overview of a family of research projects we are undertaking at… (More)
Modern user interfaces can take advantage of multiple input modalities such as speech, gestures, handwriting... to increase robustness and flexibility. The construction of such multimodal interfaces would be greatly facilitated by a unified framework that provides methods to characterize and interpret multimodal inputs. In this paper we describe a semantic… (More)
The Time Delay Neural Network (TDNN) is one of the neural network architectures that give excellent performance in tasks involving classification of temporal signals, such as phoneme classification, on-line gesture and handwriting recognition, and many others. One particular problem that occurs in on-line recognition tasks is how to deal with input patterns… (More)
In this paper, we present an adaptive multimodal interface for wireless computing applications. The system optimizes its performance by dynamically selecting its network service based on cost and performance requirements. We describe a prototype system and its application to demonstrate the proposed concept.