iToF2dToF: A Robust and Flexible Representation for Data-Driven Time-of-Flight Imaging

  title={iToF2dToF: A Robust and Flexible Representation for Data-Driven Time-of-Flight Imaging},
  author={Felipe Gutierrez-Barragan and Huaijin Chen and Mohit Gupta and Andreas Velten and Jinwei Gu},
  journal={IEEE Transactions on Computational Imaging},
Indirect Time-of-Flight (iToF) cameras are a promising depth sensing technology. However, they are prone to errors caused by multi-path interference (MPI) and low signal-to-noise ratio (SNR). Traditional methods, after denoising, mitigate MPI by estimating a transient image that encodes depths. Recently, data-driven methods that jointly denoise and mitigate MPI have become state-of-the-art without using the intermediate transient representation. In this paper, we propose to revisit the… 
Lightweight Deep Learning Architecture for MPI Correction and Transient Reconstruction
The proposed model reaches state-of-the-art MPI correction performances both on synthetic and real data and proves to be very competitive also at extreme levels of noise; at the same time, it makes a step towards reconstructing transient information from multi-frequency iToF data.
Time-of-Flight Imaging in Fog Using Polarization Phasor Imaging
Due to the light scattered by atmospheric aerosols, the amplitude image contrast is degraded and the depth measurement is greatly distorted for time-of-flight (ToF) imaging in fog. The problem limits
RADU: Ray-Aligned Depth Update Convolutions for ToF Data Denoising
This paper proposes an iterative correcting approach operating in 3D space, that is designed to learn on 2.5D data by enabling 3D point convolutions to correct the points’ positions along the view direction, and is able to outperform SOTA methods on several datasets.
UltraDepth: Exposing High-Resolution Texture from Depth Cameras
The first system that can expose high-resolution texture from depth maps captured by off-the-shelf ToF cameras, simply by introducing a distorting IR source is proposed, which offers over 10x improvement over the ordinary depth maps.


Model Study of Transient Imaging With Multi-Frequency Time-of-Flight Sensors
This paper proposes a compressive transient imaging model, without any priori knowledge, by constructing a near-tight-frame based representation of the ToF imaging principle, and proves that the compressibility of sensor measurements can be presented in the Fourier domain and held in the frame, and the tof measurements possess multi-scale characteristics.
Deep End-to-End Time-of-Flight Imaging
It is demonstrated that the proposed network can efficiently exploit the spatio-temporal structures of ToF frequency measurements, and validate the performance of the joint multipath removal, denoising and phase unwrapping method on a wide range of challenging scenes.
Bayesian Time-of-Flight for Realtime Shape, Illumination and Albedo
A generative probabilistic model that accurately relates latent imaging conditions to observed camera responses, capable of describing some multipath effects, and a physically accurate simulation of multipath phenomena is proposed.
DeepToF: off-the-shelf real-time correction of multipath interference in time-of-flight imaging
A new technique is proposed that corrects errors in depth caused by MPI that requires no camera modifications, and corrects depth in just 10 milliseconds per frame, and is demonstrated and validated on both synthetic and real complex scenarios.
Solving trigonometric moment problems for fast transient imaging
This work proposes new techniques for measurement and reconstruction of transient images, which drastically reduce this capture time, and proposes modifications of the capture routine to achieve the required sinusoidal modulation without increasing the capture time.
Reconstructing Transient Images from Single-Photon Sensors
In this work, an imaging system that builds on single photon avalanche diode (SPAD) sensors to capture multi-path responses with picosecond-scale active illumination is designed and evaluated and inverse methods that use modern approaches to deconvolve and denoise measurements in the presence of Poisson noise are developed.
Ellipsoidal path connections for time-gated rendering
This work introduces a procedure for efficiently sampling paths with a predetermined length, and incorporates it within rendering frameworks tailored towards simulating time-gated imaging, to empirically demonstrate improved rendering performance in a variety of applications.
Deep Learning for Multi-path Error Removal in ToF Sensors
A novel method for MPI removal and depth refinement exploiting an ad-hoc deep learning architecture working on data from a multi-frequency ToF camera using a Convolutional Neural Network made of two sub-networks.
Auto-Tuning Structured Light by Optical Stochastic Gradient Descent
The key idea is to formulate a stochastic gradient descent (SGD) optimization procedure that puts the actual system in the loop: projecting patterns, capturing images, and calculating the gradient of expected reconstruction error.
Deep End-to-End Alignment and Refinement for Time-of-Flight RGB-D Module
This work proposes a framework for jointly alignment and refinement via deep learning, achieving state-of-the-art for ToF refinement through improved kernel predicting network that performs kernel normalization and applies the bias prior to the dynamic convolution.