Tangent Space Backpropagation for 3D Transformation Groups

@article{Teed2021TangentSB,
  title={Tangent Space Backpropagation for 3D Transformation Groups},
  author={Zachary Teed and Jia Deng},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={10333-10342}
}
  • Zachary Teed, Jia Deng
  • Published 22 March 2021
  • Mathematics, Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
We address the problem of performing backpropagation for computation graphs involving 3D transformation groups SO(3), SE(3), and Sim(3). 3D transformation groups are widely used in 3D vision and robotics, but they do not form vector spaces and instead lie on smooth manifolds. The standard backpropagation approach, which embeds 3D transformations in Euclidean spaces, suffers from numerical difficulties. We introduce a new library, which exploits the group structure of 3D transformations and… 
View on IEEE
View With Semantic Reader
6 Citations
Background Citations
1
Methods Citations
4

Figures and Tables from this paper

6 Citations

Citation Type

Citation Type

Projective Manifold Gradient Layer for Deep Rotation Regression
TLDR
The proposed regularized projective manifold gradient (RPMG) method helps networks achieve new state-of-the-art performance in a variety of rotation estimation tasks and can be applied to other smooth manifolds such as the unit sphere.
Vision-Only Robot Navigation in a Neural Radiance World
TLDR
This work proposes an algorithm for navigating a robot through a 3D environment represented as a NeRF using only an onboard RGB camera for localization, and introduces a trajectory optimization algorithm that avoids collisions with high-density regions in the NeRF based on a discrete time version of differential flatness.
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
TLDR
NICE-SLAM is presented, a dense SLAM system that incorporates multi-level local information by introducing a hierarchical scene representation and optimizing this representation with pre-trained geometric priors enables detailed reconstruction on large indoor scenes.
DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras
TLDR
This work introduces DROID-SLAM, a new deep learning based SLAM system that is accurate, achieving large improvements over prior work, and robust, suffering from substantially fewer catastrophic failures.
SymForce: Symbolic Computation and Code Generation for Robotics
TLDR
Novel methods to automatically compute tangent space Jacobians are introduced, eliminating the need for bug-prone handwritten derivatives, which enables faster runtime code, faster development time, and fewer lines of handwritten code versus the state-of-the-art.
RAFT-3D: Scene Flow using Rigid-Motion Embeddings
  • Zachary Teed, Jia Deng
  • Computer Science
    2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2021
TLDR
RAFT-3D is introduced, a new deep architecture for scene flow based on the RAFT model developed for optical flow but iteratively updates a dense field of pixelwise SE3 motion instead of 2D motion, which represents a soft grouping of pixels into rigid objects.

References

SHOWING 1-10 OF 42 REFERENCES
Manopt, a matlab toolbox for optimization on manifolds
TLDR
The Manopt toolbox, available at www.manopt.org, is a user-friendly, documented piece of software dedicated to simplify experimenting with state of the art Riemannian optimization algorithms, which aims particularly at lowering the entrance barrier.
Manifold Geometry with Fast Automatic Derivatives and Coordinate Frame Semantics Checking in C++
Computer vision and robotics problems often require representation and estimation of poses on the SE(3) manifold. Developers of algorithms that must run in real time face several time-consuming…
Initialization techniques for 3D SLAM: A survey on rotation estimation and its use in pose graph optimization
TLDR
It is shown that the use of rotation estimation to bootstrap iterative pose graph solvers entails significant boost in convergence speed and robustness.
SuperGlue: Learning Feature Matching With Graph Neural Networks
TLDR
SuperGlue is introduced, a neural network that matches two sets of local features by jointly finding correspondences and rejecting non-matchable points and introduces a flexible context aggregation mechanism based on attention, enabling SuperGlue to reason about the underlying 3D scene and feature assignments jointly.
A micro Lie theory for state estimation in robotics
TLDR
This paper will walk through the most basic principles of the Lie theory, with the aim of conveying clear and useful ideas, and leave a significant corpus of theLie theory behind.
g 2 o: A general Framework for (Hyper) Graph Optimization
TLDR
A C++ framework for performing the optimization of nonlinear least squares problems that can be embedded as a graph or in an hyper-graph, where go stands for General (Hyper) Graph Optimization.
Distributed 3-D Localization of Camera Sensor Networks From 2-D Image Measurements
TLDR
Distributed algorithms that use 2-D image measurements to estimate the absolute 3-D poses of the nodes in a camera network, with the purpose of enabling higher-level tasks such as tracking and recognition are proposed.
Factor Graphs and GTSAM: A Hands-on Introduction
TLDR
This document provides a hands-on introduction to both factor graphs and GTSAM, a BSD-licensed C++ library based on factor graphs developed at the Georgia Institute of Technology by myself, many of my students, and collaborators.
SE3-nets: Learning rigid body motion using deep neural networks
TLDR
SE3-Nets which are deep neural networks designed to model and learn rigid body motion from raw point cloud data and predict SE(3) transformations for different parts of the scene are introduced.
DeepFactors: Real-Time Probabilistic Dense Monocular SLAM
TLDR
A SLAM system that unifies these methods in a probabilistic framework while still maintaining real-time performance through the use of a learned compact depth map representation and reformulating three different types of errors: photometric, reprojection and geometric.
...
1
2
3
4
5
...