DFNet: Enhance Absolute Pose Regression with Direct Feature Matching

  title={DFNet: Enhance Absolute Pose Regression with Direct Feature Matching},
  author={Shuai Chen and Xinghui Li and Zirui Wang and Victor Adrian Prisacariu},
We introduce a camera relocalization pipeline that combines absolute pose regression (APR) and direct feature matching. Existing photometric-based methods have trouble on scenes with large photometric distortions, e.g. outdoor environments. By incorporating an exposure-adaptive novel view synthesis, our methods can successfully address the challenges. Moreover, by introducing domain-invariant feature matching, our solution can improve pose regression accuracy while using semi-supervised… 
1 Citations

Figures and Tables from this paper

LATITUDE: Robotic Global Localization with Truncated Dynamic Low-pass Filter in City-scale NeRF

LATITUDE: Global Localization with Truncated Dynamic Low-pass Filter, which introduces a two-stage localization mechanism in city-scale NeRF, and evaluates the method on both synthetic and real-world data and shows its potential applications for high-precision navigation in large- scale city scenes.



Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

This work designs a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering and shows that the method can easily cope with additional unlabeled data without the need for external supervision.

Reassessing the Limitations of CNN Methods for Camera Pose Regression

This paper proposes a model that can directly regress the camera pose from images with significantly higher accuracy than existing methods of the same class and proposes a way to overcome the biased training data by a novel training technique, which generates poses guided by a probability distribution from the training set for synthesising new training views.

Geometric Loss Functions for Camera Pose Regression with Deep Learning

  • Alex KendallR. Cipolla
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
A number of novel loss functions for learning camera pose which are based on geometry and scene reprojection error are explored, and it is shown how to automatically learn an optimal weighting to simultaneously regress position and orientation.

Paying Attention to Activation Maps in Camera Pose Regression

This work proposes an attention-based approach for pose regression, where the convolutional activation maps are used as sequential inputs, and Transformers are applied to encode the sequential activation maps as latent vectors, used for camera pose regression.

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

This work trains a convolutional neural network to regress the 6-DOF camera pose from a single RGB image in an end-to-end manner with no need of additional engineering or graph optimisation, demonstrating that convnets can be used to solve complicated out of image plane regression problems.

Synthetic View Generation for Absolute Pose Regression and Image Synthesis

This work proposes a synthetic data augmentation technique and design a deep neural network, that can be trained to estimate the absolute pose of an image from synthesized sparse feature descriptors, that surpasses recent deep learning based synthetic image generation technique.

Back to the Feature: Learning Robust Camera Localization from Pixels to Pose

PixLoc is introduced, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model, based on the direct alignment of multiscale deep features, casting camera localization as metric learning.

Image-Based Localization Using LSTMs for Structured Feature Correlation

Experimental results show the proposed CNN+LSTM architecture for camera pose regression for indoor and outdoor scenes outperforms existing deep architectures, and can localize images in hard conditions, where classic SIFT-based methods fail.

LENS: Localization enhanced by NeRF synthesis

This paper demonstrates improvement of camera pose regression thanks to an additional synthetic dataset rendered by the NeRF class of algorithm and improves localization accuracy of pose regressors using synthesized realistic and geometry consistent images as data augmentation during training.

Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images

We address the problem of inferring the pose of an RGB-D camera relative to a known 3D scene, given only a single acquired image. Our approach employs a regression forest that is capable of inferring