Tangent Space Backpropagation for 3D Transformation Groups

  title={Tangent Space Backpropagation for 3D Transformation Groups},
  author={Zachary Teed and Jia Deng},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  • Zachary Teed, Jia Deng
  • Published 22 March 2021
  • Mathematics, Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
We address the problem of performing backpropagation for computation graphs involving 3D transformation groups SO(3), SE(3), and Sim(3). 3D transformation groups are widely used in 3D vision and robotics, but they do not form vector spaces and instead lie on smooth manifolds. The standard backpropagation approach, which embeds 3D transformations in Euclidean spaces, suffers from numerical difficulties. We introduce a new library, which exploits the group structure of 3D transformations and… 

Figures and Tables from this paper

Projective Manifold Gradient Layer for Deep Rotation Regression
The proposed regularized projective manifold gradient (RPMG) method helps networks achieve new state-of-the-art performance in a variety of rotation estimation tasks and can be applied to other smooth manifolds such as the unit sphere.
Vision-Only Robot Navigation in a Neural Radiance World
This work proposes an algorithm for navigating a robot through a 3D environment represented as a NeRF using only an onboard RGB camera for localization, and introduces a trajectory optimization algorithm that avoids collisions with high-density regions in the NeRF based on a discrete time version of differential flatness.
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
NICE-SLAM is presented, a dense SLAM system that incorporates multi-level local information by introducing a hierarchical scene representation and optimizing this representation with pre-trained geometric priors enables detailed reconstruction on large indoor scenes.
DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras
This work introduces DROID-SLAM, a new deep learning based SLAM system that is accurate, achieving large improvements over prior work, and robust, suffering from substantially fewer catastrophic failures.
SymForce: Symbolic Computation and Code Generation for Robotics
Novel methods to automatically compute tangent space Jacobians are introduced, eliminating the need for bug-prone handwritten derivatives, which enables faster runtime code, faster development time, and fewer lines of handwritten code versus the state-of-the-art.
RAFT-3D: Scene Flow using Rigid-Motion Embeddings
  • Zachary Teed, Jia Deng
  • Computer Science
    2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2021
RAFT-3D is introduced, a new deep architecture for scene flow based on the RAFT model developed for optical flow but iteratively updates a dense field of pixelwise SE3 motion instead of 2D motion, which represents a soft grouping of pixels into rigid objects.


Manopt, a matlab toolbox for optimization on manifolds
The Manopt toolbox, available at www.manopt.org, is a user-friendly, documented piece of software dedicated to simplify experimenting with state of the art Riemannian optimization algorithms, which aims particularly at lowering the entrance barrier.
Manifold Geometry with Fast Automatic Derivatives and Coordinate Frame Semantics Checking in C++
Computer vision and robotics problems often require representation and estimation of poses on the SE(3) manifold. Developers of algorithms that must run in real time face several time-consuming
Initialization techniques for 3D SLAM: A survey on rotation estimation and its use in pose graph optimization
It is shown that the use of rotation estimation to bootstrap iterative pose graph solvers entails significant boost in convergence speed and robustness.
SuperGlue: Learning Feature Matching With Graph Neural Networks
SuperGlue is introduced, a neural network that matches two sets of local features by jointly finding correspondences and rejecting non-matchable points and introduces a flexible context aggregation mechanism based on attention, enabling SuperGlue to reason about the underlying 3D scene and feature assignments jointly.
A micro Lie theory for state estimation in robotics
This paper will walk through the most basic principles of the Lie theory, with the aim of conveying clear and useful ideas, and leave a significant corpus of theLie theory behind.
g 2 o: A general Framework for (Hyper) Graph Optimization
A C++ framework for performing the optimization of nonlinear least squares problems that can be embedded as a graph or in an hyper-graph, where go stands for General (Hyper) Graph Optimization.
Distributed 3-D Localization of Camera Sensor Networks From 2-D Image Measurements
Distributed algorithms that use 2-D image measurements to estimate the absolute 3-D poses of the nodes in a camera network, with the purpose of enabling higher-level tasks such as tracking and recognition are proposed.
Factor Graphs and GTSAM: A Hands-on Introduction
This document provides a hands-on introduction to both factor graphs and GTSAM, a BSD-licensed C++ library based on factor graphs developed at the Georgia Institute of Technology by myself, many of my students, and collaborators.
SE3-nets: Learning rigid body motion using deep neural networks
SE3-Nets which are deep neural networks designed to model and learn rigid body motion from raw point cloud data and predict SE(3) transformations for different parts of the scene are introduced.
DeepFactors: Real-Time Probabilistic Dense Monocular SLAM
A SLAM system that unifies these methods in a probabilistic framework while still maintaining real-time performance through the use of a learned compact depth map representation and reformulating three different types of errors: photometric, reprojection and geometric.