Learn More
As an outgrowth of our interest in dense wireless sensing and expressive applications of wearable computing, we have developed the world's most versatile human-computer interface for the foot. By dense wireless sensing, we mean the remote acquisition of many different parameters with a compact, autonomous sensor cluster. We have developed such a low-power(More)
Lifton This paper describes four different systems that we have developed for capturing various manners of gesture near interactive surfaces. The first is a low-cost scanning laser rangefinder adapted to accurately track the position of bare hands in a plane just above a large projection display. The second is an acoustic system that detects the position of(More)
People leverage situational context when using language. Rather than convey all information through words, listeners can infer speakers' meanings due to shared common ground [1, 2]. For machines to engage fully in conversation with humans, they must also link words to the world. We present a sensorimotor representation for physically grounding action verbs,(More)
An interactive environment has been developed that uses a pair of Doppler radars to measure upper-body kinematics (velocity, direction of motion, amount of motion) and a grid of piezoelectric wires hidden under a 6 x 10 foot carpet to monitor dynamic foot position and pressure. This system has been used in an audio installation, where users launch and(More)
To build robots that engage in fluid face-to-face spoken conversations with people, robots must have ways to connect what they say to what they see. A critical aspect of how language connects to vision is that language encodes points of view. The meaning of my left and your left differs due to an implied shift of visual perspective. The connection of(More)
— Human cognition makes extensive use of visu-alization and imagination. As a first step towards giving a robot similar abilities, we have built a robotic system that uses a perceptually-coupled physical simulator to produce an internal world model of the robot's environment. Real-time perceptual coupling ensures that the model is constantly kept in(More)
We introduce an approach for physically-grounded natural language interpretation by robots which reacts appropriately to unanticipated physical changes in the environment and dynamically assimilates new information pertinent to ongoing tasks. At the core of the approach is a model of object schemas that enables a robot to encode beliefs about physical(More)