• Publications
  • Influence
KinectFusion: Real-time dense surface mapping and tracking
We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware.Expand
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Novel extensions to the core GPU pipeline demonstrate object segmentation and user interaction directly in front of the sensor, without degrading camera tracking or reconstruction, to enable real-time multi-touch interactions anywhere. Expand
Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum
A novel machine learning architecture, specifically designed for radio-frequency based gesture recognition, based on an end-to-end trained combination of deep convolutional and recurrent neural networks, for Google's Soli sensor. Expand
Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor
Digits is a wrist-worn sensor that recovers the full 3D pose of the user's hand, which enables a variety of freehand interactions on the move and is specifically designed to be low-power and easily reproducible using only off-the-shelf hardware. Expand
Cross-Modal Deep Variational Hand Pose Estimation
This work proposes a method to learn a statistical hand model represented by a cross-modal trained latent space via a generative deep neural network, which can be directly used to estimate 3D hand poses from RGB images, outperforming the state-of-the art in different settings. Expand
HoloDesk: direct 3d interactions with a situated see-through display
A new technique for interpreting raw Kinect data is introduced to approximate and track rigid and non-rigid physical objects and support a variety of physics-inspired interactions between virtual and real. Expand
KinectFusion: real-time dynamic 3D surface reconstruction and interaction
We present KinectFusion, a system that takes live depth data from a moving Kinect camera and in real-time creates high-quality, geometrically accurate, 3D models. Our system allows a user holding aExpand
Interactions in the air: adding further depth to interactive tabletops
The goal is to design a technique that closely resembles the ways the authors manipulate physical objects in the real-world; conceptually, allowing virtual objects to be 'picked up' off the tabletop surface in order to manipulate their three dimensional position or orientation. Expand
Bringing physics to the surface
This paper defines a technique for modeling the data sensed from such surfaces as input within a physics simulation, capable of modeling both multiple contact points and more sophisticated shape information, and of mapping this user input to contact forces due to friction and collisions within the physics simulation. Expand
Learning Human Motion Models for Long-Term Predictions
The Dropout Autoencoder LSTM (DAELSTM), a new architecture for the learning of predictive spatio-temporal motion models from data alone, is capable of synthesizing natural looking motion sequences over long-time horizons without catastrophic drift or motion degradation. Expand