Tracking objects outside the line of sight using 2D intensity images

  title={Tracking objects outside the line of sight using 2D intensity images},
  author={Jonathan Klein and Christoph Peters and Jaime Mart{\'i}n and Martin Laurenzis and Matthias B. Hullin},
  journal={Scientific Reports},
The observation of objects located in inaccessible regions is a recurring challenge in a wide variety of important applications. Recent work has shown that using rare and expensive optical setups, indirect diffuse light reflections can be used to reconstruct objects and two-dimensional (2D) patterns around a corner. Here we show that occluded objects can be tracked in real time using much simpler means, namely a standard 2D camera and a laser pointer. Our method fundamentally differs from… 

Non-line-of-sight Imaging with Partial Occluders and Surface Normals

This work introduces a factored NLOS light transport representation that accounts for partial occlusions and surface normals and develops a factorization approach for inverse time-resolved light transport and demonstrates high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.

Keyhole Imaging:Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path

This work proposes a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole, and derives inverse methods based on expectation-maximization to recover the object's shape and location using these measurements.

Non-line-of-sight MoCap

This work has developed a novel analysis-by-synthesis scheme which utilizes the output of a specialized renderer as input to an optimizer to perform the reconstruction of an object hidden behind an occluder.

Two-Dimensional Non-Line-of-Sight Scene Estimation From a Single Edge Occluder

A new forward model is derived, accounting for radial falloff, and two inversion algorithms are proposed to form 2D reconstructions from a single photograph of the penumbra to demonstrate the feasibility (and utility) of the 2D corner camera.

Occlusion-based computational periscopy with consumer cameras

This work presents a physical model that relates the measured photograph to the radiosity of the hidden scene and the visibility function due to the opaque occluding object, and presents a two-step computational method for recovering the occluder and the scene behind it.

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

This article introduces a novel reconstruction scheme that, by design, produces solutions that are consistent with state-of-the-art physically based rendering and evaluation on a variety of synthetic and experimental input data shows that it gracefully handles uncooperative scenes with high levels of noise or non-diffuse material reflectance.

Objects as Cameras: Estimating High-Frequency Illumination from Shadows

High-frequency information encoded in the shadows cast by an object is recovered to estimate a hemispherical photograph from the viewpoint of the object, effectively turning objects into cameras and developing a methodology for evaluating expected recovery performance for different types of objects.

Exploiting Occlusion in Non-Line-of-Sight Active Imaging

It is demonstrated that the presence of occluders in the hidden scene can obviate the need for collecting time-resolved measurements, and an accompanying analysis for such systems and their generalizations is developed.

A Calibration Scheme for Non-Line-of-Sight Imaging Setups

This paper proposes a method for calibrating time-of-flight-based non-line- of-sight imaging systems that relies on mirrors as known targets that is general enough to be applicable to a variety of sensing scenarios ranging from single sources/detectors via scanning arrangements to large-scale arrays.

Computational periscopy with an ordinary digital camera

This work introduces a two-dimensional computational periscopy technique that requires only a single photograph captured with an ordinary digital camera to recover the position of an opaque object and the scene behind (but not completely obscured by) the object, when both the object and scene are outside the line of sight of the camera, without requiring controlled or time-varying illumination.



Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging.

A three-dimensional range camera able to look around a corner using diffusely reflected light that achieves sub-millimetre depth precision and centimetre lateral precision over 40 cm×40cm×40 cm of hidden space is demonstrated.

Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using Inexpensive Time-of-Flight Sensors

This paper forms the reconstruction task as a linear inverse problem on the transient response of a scene, which they acquire using an affordable setup consisting of a modulated light source and a time-of-flight image sensor, and achieves resolutions in the order of a few centimeters for object shape and albedo.

Detection and tracking of moving objects hidden from view

An experimental method is demonstrated that allows them to locate and track moving targets that are hidden from the direct line of sight, for example, by a wall or an obstacle, with only a few seconds acquisition time and centimetre precision.

Dual photography

A novel photographic technique called dual photography is presented, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene, and is fundamentally a more efficient way to capture such a 6D dataset than a system based on multiple projectors and one camera.

See around the corner using active imaging

This paper investigates the prospects of "seeing around the corner" using active imaging. A monostatic active imaging system offers interesting capabilities in the presence of glossy reflecting

Modeling the interaction of light between diffuse surfaces

A method is described which models the interaction of light between diffusely reflecting surfaces, which includes the effects of diffuse light sources of finite area, as well as the “color-bleeding” effects which are caused by the diffuse reflections.

Occluded Imaging with Time-of-Flight Sensors

It is concluded that off-the-shelf TOF cameras can look around corners and that performance depends on two primary factors: camera modulation frequency and the width of the specular lobe (“shininess”) of the wall.

Nonline-of-sight laser gated viewing of scattered photons

Abstract. Laser gated viewing is a prominent sensing technology for optical imaging in harsh environments and can be applied for vision through fog, smoke, and other degraded environmental conditions

Coded time of flight cameras

The key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns.

Solving trigonometric moment problems for fast transient imaging

This work proposes new techniques for measurement and reconstruction of transient images, which drastically reduce this capture time, and proposes modifications of the capture routine to achieve the required sinusoidal modulation without increasing the capture time.