TP-TIO: A Robust Thermal-Inertial Odometry with Deep ThermalPoint

  title={TP-TIO: A Robust Thermal-Inertial Odometry with Deep ThermalPoint},
  author={Shibo Zhao and Peng Wang and Hengrui Zhang and Zheng Fang and Sebastian A. Scherer},
  journal={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  • Shibo Zhao, Peng Wang, S. Scherer
  • Published 24 October 2020
  • Computer Science
  • 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
To achieve robust motion estimation in visually degraded environments, thermal odometry has been an attraction in the robotics community. However, most thermal odometry methods are purely based on classical feature extractors, which is difficult to establish robust correspondences in successive frames due to sudden photometric changes and large thermal noise. To solve this problem, we propose ThermalPoint, a lightweight feature detection network specifically tailored for producing keypoints on… 
Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments
We propose Super Odometry, a high-precision multi-modal sensor fusion framework, providing a simple but effective way to fuse multiple sensors such as LiDAR, camera, and IMU sensors and achieve
On Visual-Aided LiDAR-Inertial Odometry System in Challenging Subterranean Environments
This thesis demonstrates that a vision-aided LiDAR-inertial odometry system can provide more robust state estimation under challenging environments and expands on the depth-enhanced visual- inertial Odometry pipeline, including hardware setup, and software architecture.
Resource-aware Online Parameter Adaptation for Computationally -constrained Visual-Inertial Navigation Systems
The algorithm proposes selected changes in the vision frontend and optimization back-end of visual-inertial odometry algorithms, both prior to execution and in real-time based on an online profiling of available resources.
Visual-Thermal Camera Dataset Release and Multi-Modal Alignment without Calibration Information
This report accompanies a dataset release on visual and thermal camera data and details a procedure followed to align such multi-modal camera frames in order to provide pixel-level correspondence
Motion Primitives-based Navigation Planning using Deep Collision Prediction
A method to design a novel navigation planner exploiting a learning-based collision prediction network and a resilient small robot integrating lightweight sensing and computing resources is developed.
From SLAM to Situational Awareness: Challenges and Survey
This paper surveys the state-of-the-art robotics algorithms, analyzes the situational awareness aspects that have been covered by them, and finds that the existing robotics algorithms are still missing manifold important aspects of situational awareness.


Keyframe‐based thermal–inertial odometry
Autonomous navigation of microaerial vehicles in environments that are simultaneously GPS‐denied and visually degraded, and especially in the dark, texture‐less and dust‐ or smoke‐filled settings, is
Keyframe-based visual–inertial odometry using nonlinear optimization
This work forms a rigorously probabilistic cost function that combines reprojection errors of landmarks and inertial terms and compares the performance to an implementation of a state-of-the-art stochastic cloning sliding-window filter.
DeepTIO: A Deep Thermal-Inertial Odometry With Visual Hallucination
A Deep Neural Network model for thermal-inertial odometry (DeepTIO) is proposed by incorporating a visual hallucination network to provide the thermal network with complementary information and employs selective fusion to attentively fuse the features from three different modalities.
Direct Sparse Odometry
The experiments show that the presented approach significantly outperforms state-of-the-art direct and indirect methods in a variety of real-world settings, both in terms of tracking accuracy and robustness.
Practical Infrared Visual Odometry
This paper proposes a practical visual odometry system based on a monocular thermal camera that performs efficient ground plane detection for targeted feature extraction and addresses the problem of periodic nonuniformity correction.
Sparse Depth Enhanced Direct Thermal-Infrared SLAM Beyond the Visible Spectrum
A method to use sparse depth measurement for 6-DOF motion estimation via direct tracking under 14-bit raw measurement from the thermal camera enhanced by sparse depth measurements from light detection ranging (LiDAR) is proposed.
Robocentric Visual-Inertial Odometry
  • Zheng Huai, Guoquan Huang
  • Computer Science
    2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2018
The proposed R-VIO algorithm is extensively tested through both Monte Carlo simulations and real-world experiments and shown to achieve competitive performance with the state-of-the-art VINS algorithms in terms of consistency, accuracy and efficiency.
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
This paper presents VINS-Mono: a robust and versatile monocular visual-inertial state estimator that is applicable for different applications that require high accuracy in localization and performs an onboard closed-loop autonomous flight on the microaerial-vehicle platform.
Multispectral Stereo Odometry
This paper investigates the problem of visual odometry for ground vehicles based on the simultaneous utilization of multispectral cameras using a stereo rig composed of an optical and thermal sensors rather than two monocular cameras of different spectrums.
Multi-Spectral Visual Odometry without Explicit Stereo Matching
Experimental results indicate that the proposed method can provide accurate visual odometry results with recovered metric scale and can also provide a metric 3D reconstruction in semi-dense density with multi-spectral information, which is not available from existing multi-Spectral methods.