Spacetime stereo: a unifying framework for depth from triangulation

  title={Spacetime stereo: a unifying framework for depth from triangulation},
  author={James Davis and Ravi Ramamoorthi and Szymon M. Rusinkiewicz},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
Depth from triangulation has traditionally been investigated in a number of independent threads of research, with methods such as stereo, laser scanning, and coded structured light considered separately. We propose a common framework called spacetime stereo that unifies and generalizes many of these previous methods. To show the practical utility of the framework, we develop two new algorithms for depth estimation: depth from unstructured illumination change and depth estimation in dynamic… 

Spacetime stereo: a unifying framework for depth from triangulation

It is shown that methods derived from the spacetime stereo framework can be used to recover depth in situations in which existing methods perform poorly.

Novel dense stereo algorithms for high-quality depth estimation from images

A new invariant for stereo correspondence is introduced which allows completely arbitrary scene reflectance (bidirectional reflectance distribution functions - BRDFs) and can be used to formulate a rank constraint on stereo matching when the scene is observed by several lighting configurations in which only the lighting intensity varies.

A 3D reconstruction system based on improved spacetime stereo

This paper shows that the standard spacetime stereo approach can be improved in terms of accuracy of disparity estimation and convergence speed by adoption of suitable matching algorithms based on adaptive support windows.

Multi-Scale 3D Scene Flow from Binocular Stereo Sequences

  • Rui LiS. Sclaroff
  • Physics
    2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1
  • 2005

Spacetime stereo and its applications

This dissertation builds a dynamic face capture system that consists of off-the-shelf cameras, projectors, and computers, and proposes a new template fitting and tracking method that computes a sequence of 3D face meshes with vertex-to-vertex correspondence by combining shape and optical flow estimation.

Spacetime Stereo and 3D Flow via Binocular Spatiotemporal Orientation Analysis

  • M. SizintsevR. Wildes
  • Environmental Science
    IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2014
A novel approach to recovering estimates of 3D structure and motion of a dynamic scene from a sequence of binocular stereo images based on matching spatiotemporal orientation distributions between left and right temporal image streams, which encapsulates both local spatial and temporal structure for disparity estimation.

Variational stereo in dynamic illumination

A variational optimization formulation for finding a dense stereo correspondence field is introduced, based on multi-frame optical flow, adapted to stereo, and yields dense and robust results.

Spatiotemporal Descriptor for Wide-Baseline Stereo Reconstruction of Non-rigid and Ambiguous Scenes

This paper studies the use of temporal consistency to match appearance descriptors and handle complex ambiguities when computing dynamic depth maps from stereo using a state-of-the-art 2D dense appearance descriptor which can be applied to wide-baseline stereo setups.

Advances in 3 D Shape Acquisition

This dissertation designs a novel algorithm for the triangulation of dynamic objects, as well as a new stereo setup based on unstructured active lighting, and presents a novel sub-pixel precision refinement algorithm for stereo matches.

A Practical Stereo System based on Regularization and Texture Projection

It is shown that in scenes with moving objects spatial regularization methods are more accurate than spacetime stereo, while remaining computationally simpler, and an extension of regularization-based algorithms to the temporal domain is proposed, to further improve the performance ofregularization methods within dynamic scenes.



Spacetime stereo: a unifying framework for depth from triangulation

It is shown that methods derived from the spacetime stereo framework can be used to recover depth in situations in which existing methods perform poorly.

Spacetime stereo: shape recovery for dynamic scenes

  • Li ZhangB. CurlessS. Seitz
  • Physics
    2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings.
  • 2003
The approach is one of very few existing methods that can robustly reconstruct objects that are moving and deforming over time, achieved by use of oriented spacetime windows in the matching procedure.

Better optical triangulation through spacetime analysis

  • B. CurlessM. Levoy
  • Physics
    Proceedings of IEEE International Conference on Computer Vision
  • 1995
A new ranging method based on analyzing the time evolution of the structured light reflections of the laser stripe triangulation scanners is presented, thereby attaining significantly higher accuracy using existing technology.

Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction

This new method of multinocular stereopsis exploits Helmholtz reciprocity by choosing pairs of light source and camera positions that guarantee that the ratio of the emitted radiance to the incident irradiance is the same for corresponding points in the two images.

High-accuracy stereo depth maps using structured light

  • Daniel ScharsteinR. Szeliski
  • Computer Science
    2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings.
  • 2003
A method for acquiring high-complexity stereo image pairs with pixel-accurate correspondence information using structured light that does not require the calibration of the light sources and yields registered disparity maps between all pairs of cameras and illumination projectors.

Epipolar-plane image analysis: An approach to determining structure from motion

This article describes the application of a technique for building a three-dimensional description of a static scene from a dense sequence of images, and shows how projective duality is used to extend the analysis to a wider class of camera motions and object types that include curved and moving objects.

The Space of All Stereo Images

  • S. SeitzJiwon Kim
  • Mathematics
    Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001
  • 2001
A theory of stereo image formation is presented that enables a complete classification of all possible stereo views, including non-perspective varieties. Towards this end, the notion of epipolar

A multibaseline stereo system with active illumination and real-time image acquisition

This work describes the implementation of a parallel depth recovery scheme for a four-camera multibaseline stereo in a convergent configuration that is capable of image capture at video rate and obtain dense stereo depth data by projecting a light pattern of frequency modulated sinusoidally varying intensity onto the scene.

When Is the Shape of a Scene Unique Given Its Light-Field: A Fundamental Theorem of 3D Vision?

It is shown that stereo computed from the light-field is ambiguous if and only if the scene is radiating light of a constant intensity (and color, etc.) over an extended region.