Semantic Pose Verification for Outdoor Visual Localization with Self-supervised Contrastive Learning

@article{Orhan2022SemanticPV,
  title={Semantic Pose Verification for Outdoor Visual Localization with Self-supervised Contrastive Learning},
  author={Semih Orhan and Josechu J. Guerrero and Yalin Bastanlar},
  journal={ArXiv},
  year={2022},
  volume={abs/2203.16945}
}
Any city-scale visual localization system has to over-come long-term appearance changes, such as varying illumination conditions or seasonal changes between query and database images. Since semantic content is more robust to such changes, we exploit semantic information to improve visual localization. In our scenario, the database consists of gnomonic views generated from panoramic images (e.g. Google Street View) and query images are collected with a standard field-of-view camera at a different… 
1 Citations

Figures and Tables from this paper

Review on Panoramic Imaging and Its Applications in Scene Understanding
TLDR
This review discusses in detail the broad application prospects and great design potential of freeform surfaces, thin-plate optics, and metasurfaces in panoramic imaging, and provides a detailed analysis on how these techniques can help enhance the performance of panoroscopic imaging systems.

References

SHOWING 1-10 OF 52 REFERENCES
Semantically guided location recognition for outdoors scenes
TLDR
This work shows that semantic segmentation labeling of man-made structures can inform the traditional bag-of-visual words models to obtain proper feature weighting and improve the overall location recognition accuracy.
Self-supervising Fine-grained Region Similarities for Large-scale Image Localization
TLDR
This work proposes to self-supervise image-to-region similarities in order to fully explore the potential of difficult positive images alongside their sub-regions and outperforms state-of-the-arts on the standard localization benchmarks and shows excellent generalization capability on multiple image retrieval datasets.
Semantics-aware visual localization under challenging perceptual conditions
TLDR
This paper proposes a novel approach for learning a discriminative holistic image representation which exploits the image content to create a dense and salient scene description and shows that the learnt image representation outperforms off-the-shelf features from the deep networks and hand-crafted features.
Semantic segmentation of outdoor panoramic images
TLDR
This work built a semantic segmentation CNN model that handles distortions in panoramic images using equirectangular convolutions that outperforms an equivalent CNN model with standard convolutions and releases a pixel-level annotated outdoorPanoramic image dataset.
Semantic Match Consistency for Long-Term Visual Localization
TLDR
This paper presents a method for scoring the individual correspondences by exploiting semantic information about the query image and the scene, and shows that the localization performance can be significantly improved compared to the state-of-the-art, as evaluated on two challenging long-term localization benchmarks.
Long-Term Visual Localization Using Semantically Segmented Images
Robust cross-seasonal localization is one of the major challenges in long-term visual navigation of autonomous vehicles. In this paper, we exploit recent advances in semantic segmentation of images,
Efficient Search in a Panoramic Image Database for Long-term Visual Localization
  • Semih Orhan, Y. Bastanlar
  • Computer Science
    2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
  • 2021
TLDR
This work focuses on a localization technique that is based on image retrieval and slides a search window in the last convolutional layer belonging to the panoramic image and compute the similarity with the descriptor extracted from the query image.
Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions
TLDR
This paper introduces the first benchmark datasets specifically designed for analyzing the impact of day-night changes, weather and seasonal variations, as well as sequence-based localization approaches and the need for better local features on visual localization.
Training Semantic Descriptors for Image-Based Localization
TLDR
It is shown that localization can be performed via descriptors solely extracted from semantically segmented images, and that the localization performance of a semantic descriptor can increase up to the level of state-of-the-art RGB image based methods.
Is This the Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization
TLDR
This paper develops multiple hand-crafted as well as a trainable approach to join into the geometric-semantic verification and shows significant improvements over state-of-the-art on a very challenging indoor dataset.
...
...