“Maximizing Rigidity” Revisited: A Convex Programming Approach for Generic 3D Shape Reconstruction from Multiple Perspective Views

  title={“Maximizing Rigidity” Revisited: A Convex Programming Approach for Generic 3D Shape Reconstruction from Multiple Perspective Views},
  author={Pan Ji and Hongdong Li and Yuchao Dai and Ian D. Reid},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
  • Pan Ji, Hongdong Li, I. Reid
  • Published 17 July 2017
  • Mathematics
  • 2017 IEEE International Conference on Computer Vision (ICCV)
Rigid structure-from-motion (RSfM) and non-rigid structure-from-motion (NRSfM) have long been treated in the literature as separate (different) problems. Inspired by a previous work which solved directly for 3D scene structure by factoring the relative camera poses out, we revisit the principle of “maximizing rigidity” in structure-from-motion literature, and develop a unified theory which is applicable to both rigid and non-rigid structure reconstruction in a rigidity-agnostic way. We… 

Figures and Tables from this paper

Non-Rigid Structure from Motion
This thesis shows that relative scal for deforming objects under perspective projection can be recovered under somemild assumption about the scene and proposes a new approach for dense detailed 3D reconstruction of a complex dynamic scene from two perspective frames.
Incremental Non-Rigid Structure-from-Motion with Unknown Focal Length
This paper presents a method for incremental Non-Rigid Structure-from-Motion (NRSfM) with the perspective camera model and the isometric surface prior with unknown focal length, and proposes a method to simultaneously recover the focal length and the non-rigid shapes.
Patch-Based Non-rigid 3D Reconstruction from a Single Depth Stream
We propose an approach for 3D reconstruction and tracking of dynamic surfaces using a single depth sensor, without any prior knowledge of the scene. It is robust to rapid inter-frame motions due to
Patch-based Non-rigid 3 D Reconstruction from a Single Depth Stream Carmel
We propose an approach for 3D reconstruction and tracking of dynamic surfaces using a single depth sensor, without any prior knowledge of the scene. It is robust to rapid inter-frame motions due to
Reconstruct as Far as You Can: Consensus of Non-Rigid Reconstruction from Feasible Regions
Experimental results show that the proposed method renews the state-of-the-art for popular benchmark data under much harsher environments, i.e., narrow camera view ranges, and it can reconstruct video-based real-world data effectively for as many areas as it can without an elaborated user input.
Robust Isometric Non-Rigid Structure-from-Motion
This work proposes a three-step automatic pipeline to solve non-Rigid Structure-from-Motion robustly by exploiting isometry, and shows that the robust NRSfM method consistently outperforms existing methods on both synthetic and real datasets.
Local Non-Rigid Structure-From-Motion From Diffeomorphic Mappings
A new formulation to non-rigid structure-from-motion that only requires the deforming surface to preserve its differential structure is proposed, which is more widely applicable, more stable in noisy and sparse imaging conditions and much faster than earlier ones, while delivering similar accuracy.
Local Non-Rigid Structure-from-Motion from Locally Diffeomorphic Mappings
We propose a new formulation to the non-rigid structurefrom-motion problem that only requires the deforming surface to meaning that its differential structure is preserved. This is a much weaker
Unsupervised Monocular Depth Reconstruction of Non-Rigid Scenes
An unsupervised monocular framework for dense depth estimation of dynamic scenes, which jointly reconstructs rigid and nonrigid parts without explicitly modelling the camera motion, and derives a training objective that aims to opportunistically preserve pairwise distances between reconstructed 3D points.
A Closed-Form Solution to Local Non-Rigid Structure-from-Motion
This paper shows that, under widely applicable assumptions, a new system of equation is derived in terms of the surface normals whose two solutions can be obtained in closed-form and can easily be disambiguated locally.


Inextensible Non-Rigid Shape-from-Motion by Second-Order Cone Programming
This work shows for the first time how to construct a Second-Order Cone Programming (SOCP) problem for Non-Rigid Shape-from-Motion (NRSfM) using the Maximum-Depth Heuristic (MDH).
Non-Rigid Shape-from-Motion for Isometric Surfaces using Infinitesimal Planarity
This paper proposes a general framework to solve Non-Rigid Shape-from-Motion (NRSfM) with the perspective camera under isometric deformations and derives an analytic solution which involves convex, linear least-squares optimization only, and outperforms existing works.
Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video
This paper offers the first variational approach to the problem of dense 3D reconstruction of non-rigid surfaces from a monocular video sequence and reconstructs highly deforming smooth surfaces densely and accurately directly from video, without the need for any prior models or shape templates.
Soft Inextensibility Constraints for Template-Free Non-rigid Reconstruction
An inextensibility prior is exploited as a way to better constrain the highly ambiguous problem of non-rigid reconstruction from monocular views and formulates an energy function that includes soft inExtensibility constraints and relies on existing discrete optimisation methods to minimise it.
Linear Local Models for Monocular Reconstruction of Deformable Surfaces
  • M. Salzmann, P. Fua
  • Computer Science
    IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2011
This paper replaces the global models by linear local models for surface patches, which can be assembled to represent arbitrary surface shapes as long as they are made of the same material.
Non-rigid structure from locally-rigid motion
Local-rigid motion is introduced, a general framework for solving the M-point, N-view structure-from-motion problem for unknown bodies deforming under orthography, providing a “soup” of specific, plausibly rigid, 3D triangles.
Multi-view structure computation without explicitly estimating motion
  • Hongdong Li
  • Computer Science
    2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • 2010
A novel scheme which bypasses the motion-estimation step, and goes directly to structure computation step is presented, demonstrating that such a scheme is not only theoretically possible, but also technically feasible and effective.
Monocular Template-based Reconstruction of Inextensible Surfaces
A monocular 3D reconstruction algorithm for inextensible deformable surfaces that uses point correspondences between a single image of the deformed surface taken by a camera with known intrinsic parameters and a template to recover the 3D surface shape as seen in the image.
Maximizing Rigidity: The Incremental Recovery of 3-D Structure from Rigid and Nonrigid Motion
It is shown how the 3-D structure of rigid as well as nonrigid objects can be recovered by maintaining an internal model of the viewed object and modifying it at each instant by the minimal nonrigidity change that is sufficient to account for the observed transformation.
Template-Free 3D Reconstruction of Poorly-Textured Nonrigid Surfaces
This paper introduces a template-free approach to reconstructing a poorly-textured, deformable surface and leverage surface isometry and formulate 3D reconstruction as the joint problem of non-rigid image registration and depth estimation.