CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection

@article{Nabati2021CenterFusionCR,
  title={CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection},
  author={Ramin Nabati and Hairong Qi},
  journal={2021 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  year={2021},
  pages={1526-1535}
}
  • Ramin Nabati, H. Qi
  • Published 10 November 2020
  • Computer Science
  • 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
The perception system in autonomous vehicles is responsible for detecting and tracking the surrounding objects. This is usually done by taking advantage of several sensing modalities to increase robustness and accuracy, which makes sensor fusion a crucial part of the perception system. In this paper, we focus on the problem of radar and camera sensor fusion and propose a middle-fusion approach to exploit both radar and camera data for 3D object detection. Our approach, called CenterFusion… 
View on IEEE
View With Semantic Reader
42 Citations
Highly Influential Citations
5
Background Citations
15
Methods Citations
21
Results Citations
1

Figures and Tables from this paper

42 Citations

BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection
TLDR
BEVDet4D with robust generalization performance reduces the velocity error and makes the vision-based methods, for the first time, become comparable with those relied on LiDAR or radar in this aspect.
Understanding the Scene: Identifying the Proper Sensor Mix in Different Weather Conditions
TLDR
This paper presents an extensive study of perception systems under different weather conditions, using real-life datasets (nuScenes and the CADCD) and identifies a set of evaluation metrics and study the quality of data from different sensors in different scenarios and conditions.
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
TLDR
The first unified endto-end sensor fusion framework for 3D detection, named FUTR3D, which can be used in (almost) any sensor configuration, is proposed, which achieves great flexibility with different sensor configurations and enables low-cost autonomous driving.
Probabilistic and Geometric Depth: Detecting Objects in Perspective
TLDR
This method obtains significant improvements on KITTI and nuScenes benchmarks, achieving the 1st place out of all monocular vision-only methods while still maintaining real-time efficiency.
CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking
TLDR
This work proposes an end-to-end network for joint object detection and tracking based on radar and camera sensor fusion and utilizes a greedy algorithm for object association, which is very suitable for autonomous driving applications.
High-level camera-LiDAR fusion for 3D object detection with machine learning
TLDR
This framework uses a Machine Learning (ML) pipeline on a combination of monocular camera and LiDAR data to detect vehicles in the surrounding 3D space of a moving platform to demonstrate an efficient and accurate inference on a validation set.
Boosting Single-Frame 3D Object Detection by Simulating Multi-Frame Point Clouds
To boost a detector for single-frame 3D object detection, we present a new approach to train it to simulate features and responses following a detector trained on multi-frame point clouds. Our…
Towards a Multi-Pixel Time-of-Flight Indoor Navigation System for Nano-Drone Applications
TLDR
This paper explores and characterizes a multi-zone Time of Flight (ToF) sensor to enhance autonomous navigation with a significantly lower computational load than most common visual-based solutions and proposes a lightweight approach to calculate collision probability from each ToF sensor frame.
HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection
TLDR
This work focuses on 2D object detection, a fundamental high-level task which is defined on the 2D image domain, and proposes HRFuser, a multi-resolution sensor fusion architecture that scales straightforwardly to an arbitrary number of input modalities.
3D Object Detection for Autonomous Driving: A Review and New Outlooks
TLDR
This paper conducts a comprehensive survey of the progress in 3D object detection from the aspects of models and sensory inputs, including LiDAR-based, camera- based, and multi-modal detection approaches, and provides an in-depth analysis of the potentials and challenges in each category of methods.
...
...

References

SHOWING 1-10 OF 36 REFERENCES
Radar-Camera Sensor Fusion for Joint Object Detection and Distance Estimation in Autonomous Vehicles
TLDR
A novel radar-camera sensor fusion framework for accurate object detection and distance estimation in autonomous driving scenarios using a middle-fusion approach to fuse the radar point clouds and RGB images is presented.
A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection
TLDR
The proposed CameraRadarFusion Net (CRF-Net) automatically learns at which level the fusion of the sensor data is most beneficial for the detection result, and is able to outperform a state-of-the-art image-only network for two different datasets.
RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects
TLDR
The proposed approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion, which learn from data to exploit both geometric and dynamic information of Radar data, and shows that exploiting Radar improves the perception capabilities of detecting faraway objects and understanding the motion of dynamic objects.
nuScenes: A Multimodal Dataset for Autonomous Driving
Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object…
RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles
  • Ramin Nabati, H. Qi
  • Computer Science
    2019 IEEE International Conference on Image Processing (ICIP)
  • 2019
TLDR
RRPN is introduced, a Radar-based real-time region proposal algorithm for object detection in autonomous driving vehicles that operates more than 100× faster while at the same time achieves higher detection precision and recall.
3D object tracking using RGB and LIDAR data
TLDR
A 3D object tracking algorithm using a 3D-LIDAR, an RGB camera and INS (GPS/IMU) sensors data, and the ego-vehicle's localization data is proposed that outputs the trajectory of the tracked object, an estimation of its current velocity, and its predicted location in the 3D world coordinate system in the next time-step.
Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation
TLDR
An extension to LaserNet, an efficient and state-of-the-art LiDAR based 3D object detector, is presented and a method for fusing image data with the LiDar data is proposed and shown to improve the detection performance of the model especially at long ranges.
InfoFocus: 3D Object Detection for Autonomous Driving with Dynamic Information Modeling
Real-time 3D object detection is crucial for autonomous cars. Achieving promising performance with high efficiency, voxel-based approaches have received considerable attention. However, previous…
PIXOR: Real-time 3D Object Detection from Point Clouds
TLDR
PIXOR is proposed, a proposal-free, single-stage detector that outputs oriented 3D object estimates decoded from pixel-wise neural network predictions that surpasses other state-of-the-art methods notably in terms of Average Precision (AP), while still runs at 10 FPS.
Joint 3D Proposal Generation and Object Detection from View Aggregation
TLDR
This work presents AVOD, an Aggregate View Object Detection network for autonomous driving scenarios that uses LIDAR point clouds and RGB images to generate features that are shared by two subnetworks: a region proposal network (RPN) and a second stage detector network.
...
...