• Publications
  • Influence
StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction
This paper presents StereoNet, the first end-to-end deep architecture for real-time stereo matching that runs at 60fps on an NVidia Titan X, producing high-quality, edge-preserved, quantization-free
Fusion4D: real-time performance capture of challenging scenes
TLDR
This work contributes a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time, highly robust to both large frame-to-frame motion and topology changes, allowing us to reconstruct extremely challenging scenes.
Holoportation: Virtual 3D Teleportation in Real-time
TLDR
This paper demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras, and allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space.
ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems
TLDR
This paper presents ActiveStereoNet, the first deep learning solution for active stereo systems that is fully self-supervised, yet it produces precise depth with a subpixel precision; it does not suffer from the common over-smoothing issues; it preserves the edges; and it explicitly handles occlusions.
In-air gestures around unmodified mobile devices
TLDR
A novel machine learning based algorithm extending the interaction space around mobile devices to augment and enrich the existing interaction vocabulary using gestures, which removes screen real-estate issues on small touchscreens, and allows input to be expanded to the 3D space around the device.
Learning to be a depth camera for close-range human capture and interaction
TLDR
This work presents a machine learning technique for estimating absolute, per-pixel depth using any conventional monocular 2D camera, with minor hardware modifications, and uses hybrid classification-regression forests to learn how to map from near infrared intensity images to absolute, metric depth in real-time.
State of the Art on Neural Rendering
TLDR
This state‐of‐the‐art report summarizes the recent trends and applications of neural rendering and focuses on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photorealistic outputs.
Keep it simple and sparse: real-time action recognition
TLDR
The main contribution of the paper is an effective real-time system for one-shot action modeling and recognition; the paper highlights the effectiveness of sparse coding techniques to represent 3D actions.
Motion2fusion: real-time volumetric performance capture
TLDR
This work provides three major contributions over prior work: a new non-rigid fusion pipeline allowing for far more faithful reconstruction of high frequency geometric details, avoiding the over-smoothing and visual artifacts observed previously, a high speed pipeline coupled with a machine learning technique for 3D correspondence field estimation reducing tracking errors and artifacts that are attributed to fast motions.
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
TLDR
HITNet is a novel neural network architecture for real-time stereo matching that not only geometrically reasons about disparities but also infers slanted plane hypotheses allowing to more accurately perform geometric warping and upsampling operations.
...
...