PST900: RGB-Thermal Calibration, Dataset and Segmentation Network

@article{Shivakumar2020PST900RC,
  title={PST900: RGB-Thermal Calibration, Dataset and Segmentation Network},
  author={S. S. Shivakumar and Neil Rodrigues and Alex Zhou and Ian D. Miller and Vijay R. Kumar and C. J. Taylor},
  journal={2020 IEEE International Conference on Robotics and Automation (ICRA)},
  year={2020},
  pages={9441-9447}
}
In this work we propose long wave infrared (LWIR) imagery as a viable supporting modality for semantic segmentation using learning-based techniques. We first address the problem of RGB-thermal camera calibration by proposing a passive calibration target and procedure that is both portable and easy to use. Second, we present PST900, a dataset of 894 synchronized and calibrated RGB and Thermal image pairs with per pixel human annotations across four distinct classes from the DARPA Subterranean… Expand
HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images
TLDR
This work proposes a multimodal semantic segmentation model that can be applied during daytime and nighttime, and presents a new dataset comprising over 20,000 time-synchronized and aligned RGB-thermal image pairs. Expand
Weakly Supervised Instance Segmentation of Electrical Equipment Based on RGB-T Automatic Annotation
TLDR
A progressively optimized model (POM) for instance segmentation is proposed, which first utilizes the fully connected conditional random field (CRF) and the constrain-to-boundary loss to specify fine-detailed boundaries of each object and to solve the difficulty of segmenting electrical equipment with complicated structures. Expand
MCNet: Multi-level Correction Network for thermal image semantic segmentation of nighttime driving scene
TLDR
A multi-level correction network (MCNet) with amulti-level attention module (MAM), a multi- level edge enhancement module (MEEM), and a new dataset called SCUT-Seg which contains 2010 thermal images recorded from various road scenes with 10 manually annotated semantic region labels are presented. Expand
ABMDRNet: Adaptive-Weighted Bi-Directional Modality Difference Reduction Network for RGB-T Semantic Segmentation
  • Qiang Zhang, Shenlu Zhao, Yongjiang Luo, Dingwen Zhang, Nianchang Huang, Jungong Han
  • Computer Science
  • CVPR
  • 2021
Semantic segmentation models gain robustness against poor lighting conditions by virtue of complementary information from visible (RGB) and thermal images. Despite its importance, most existing RGB-TExpand
Unsupervised Depth and Ego-motion Estimation for Monocular Thermal Video using Multi-spectral Consistency Loss
TLDR
This is the first work to simultaneously estimate both depth and ego-motion from the monocular thermal video in an unsupervised manner, and gives complementary supervision for the networks by reconstructing visible and thermal images with the depth and pose estimated from thermal images. Expand
Point Cloud Hand–Object Segmentation Using Multimodal Imaging with Thermal and Color Data for Safe Robotic Object Handover
This paper presents an application of neural networks operating on multimodal 3D data (3D point cloud, RGB, thermal) to effectively and precisely segment human hands and objects held in hand toExpand
A Dataset for Evaluating Multi-spectral Motion Estimation Methods
TLDR
A novel dataset for evaluating the performance of multi-spectral motion estimation systems is presented, which includes both multi-Spectral and dense depth images with accurate ground-truth camera poses provided by a motion capture system. Expand
Far-Infrared Object Segmentation Focus on Transmission of Overall Semantic Information
TLDR
This article proposes a far-infrared object segmentation algorithm using deep learning to solve the effects caused by the blurring of infrared objects, and designs a feature enhancement module for the multi-receptive field feature map that filters out the most versatile and highly semantic feature channels to reduce the effect of temperature inversion on segmentation. Expand
Towards a Multispectral RGB-IR-UV-D Vision System - Seeing the Invisible in 3D
In this paper, we present the development of a sensing system with the capability to compute multispectral point clouds in real-time. The proposed multi-eye sensor system effectively registersExpand
Noncontact Sensing of Contagion
TLDR
The clinical literature on diagnosis of COVID-19 through symptoms that might be remotely detected as of early May 2020 shows that visible spectrum and thermal spectrum cameras offer the best options for truly noncontact sensing of those studied to date. Expand
...
1
2
...

References

SHOWING 1-10 OF 39 REFERENCES
MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes
TLDR
This work addresses the semantic segmentation of images of street scenes for autonomous vehicles based on a new RGB-Thermal dataset, which is introduced in this paper and showed that the segmentation accuracy was significantly increased by adding thermal infrared information. Expand
RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes
TLDR
This work takes the advantage of thermal images and fuse both the RGB and thermal information in a novel deep neural network that outperforms the state of the arts in semantic segmentation for autonomous vehicles. Expand
Scene Semantic Reconstruction from Egocentric RGB-D-Thermal Videos
TLDR
The goal is to leverage a multimodal sensory stream composed of RGB, depth, and thermal signals and use this data stream as an input to a new framework for joint 6 DOF camera localization, 3D reconstruction, and semantic segmentation. Expand
Siamese CNNs for RGB-LWIR Disparity Estimation
TLDR
This work developed an architecture made of two subnetworks, each consisting of the same siamese network, but taking different image patches as input, that helps achieve better performance when compared to a single subnetwork by itself. Expand
Multimodal Sensor Fusion In Single Thermal image Super-Resolution
TLDR
A multimodal, visual-thermal fusion model that addresses thermal image super-resolution, via integrating high-frequency information from the visual image, is introduced and shown to perform better against state-of-the-arts tests. Expand
Thermal Image Enhancement using Convolutional Neural Network
TLDR
This work proposes the first thermal image enhancement method based on CNN guided on RGB data, called TEN, which directly learns an end-to-end mapping a single low resolution image to the desired high resolution image. Expand
Understanding Convolution for Semantic Segmentation
TLDR
DUC is designed to generate pixel-level prediction, which is able to capture and decode more detailed information that is generally missing in bilinear upsampling, and a hybrid dilated convolution (HDC) framework in the encoding phase is proposed. Expand
Fast-SCNN: Fast Semantic Segmentation Network
TLDR
This paper introduces fast segmentation convolutional neural network (Fast-SCNN), an above real-time semantic segmentation model on high resolution image data suited to efficient computation on embedded devices with low memory. Expand
ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation
TLDR
A deep architecture that is able to run in real time while providing accurate semantic segmentation, and a novel layer that uses residual connections and factorized convolutions in order to remain efficient while retaining remarkable accuracy is proposed. Expand
MAVNet: An Effective Semantic Segmentation Micro-Network for MAV-Based Tasks
TLDR
This letter proposes MAVNet: a small, light-weight, deep neural network for real-time semantic segmentation on micro aerial vehicles (MAVs), inspired by ERFNet, which features 400 times fewer parameters and achieves comparable performance with some reference models in empirical experiments. Expand
...
1
2
3
4
...