Frustum PointNets for 3D Object Detection from RGB-D Data

@article{Qi2018FrustumPF,
  title={Frustum PointNets for 3D Object Detection from RGB-D Data},
  author={C. Qi and W. Liu and Chenxia Wu and Hao Su and L. Guibas},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={918-927}
}
  • C. Qi, W. Liu, +2 authors L. Guibas
  • Published 2018
  • Computer Science
  • 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
In this work, we study 3D object detection from RGBD data in both indoor and outdoor scenes. [...] Key Method Instead of solely relying on 3D proposals, our method leverages both mature 2D object detectors and advanced 3D deep learning for object localization, achieving efficiency as well as high recall for even small objects. Benefited from learning directly in raw point clouds, our method is also able to precisely estimate 3D bounding boxes even under strong occlusion or with very sparse points. Evaluated on…Expand
Frustum VoxNet for 3D object detection from RGB-D or Depth images
  • X. Shen, I. Stamos
  • Computer Science, Engineering
  • 2020 IEEE Winter Conference on Applications of Computer Vision (WACV)
  • 2020
TLDR
Results on SUN RGB-D dataset show that the new 3D object detection system, which is based on a small network, can process 20 frames per second with comparable detection results to the state-of-the-art, achieving a 2× speedup. Expand
3D Object Detection with Sparse Sampling Neural Networks
The advent of inexpensive 3D sensors has resulted in an abundance of 3D pointclouds and datasets. For instance, RGB-D sensors such as Kinect can result in 3D point clouds by projecting 2D pixels intoExpand
USING EDGECONV TO IMPROVE 3D OBJECT DETECTION FROM RGB-D DATA
TLDR
A novel 3D deep learning model for object localization and object bounding boxes estimation is proposed and it is indicated that the method achieved the higher mean average precision and better IOU of boundingboxes in SUN RGB-D dataset and KITTI benchmark. Expand
ImVoteNet: Boosting 3D Object Detection in Point Clouds With Image Votes
TLDR
This work builds on top of VoteNet and proposes a 3D detection architecture called ImVoteNet specialized for RGB-D scenes, based on fusing 2D votes in images and 3D Votes in point clouds, advancing state-of-the-art results by 5.7 mAP. Expand
Deep Hough Voting for 3D Object Detection in Point Clouds
TLDR
This work proposes VoteNet, an end-to-end 3D object detection network based on a synergy of deep point set networks and Hough voting that achieves state-of-the-art 3D detection on two large datasets of real 3D scans, ScanNet and SUN RGB-D with a simple design, compact model size and high efficiency. Expand
Accurate detection and 3D localization of humans using a novel YOLO-based RGB-D fusion approach and synthetic training data
TLDR
This work proposes an image-based detection approach which extends the YOLO v3 architecture with a 3D centroid loss and mid-level feature fusion to exploit complementary information from both modalities to improve 3D localization accuracy. Expand
3D Object Detection Using Scale Invariant and Feature Reweighting Networks
TLDR
A new network architecture which focuses on utilizing the front view images and frustum point clouds to generate 3D detection results and achieves better performance than the state-of-the-art methods especially when point clouds are highly sparse. Expand
Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving
TLDR
This paper proposes a monocular 3D object detection framework in the domain of autonomous driving, and proposes a multi-modal feature fusion module to embed the complementary RGB cue into the generated point clouds representation. Expand
Learning to Predict the 3D Layout of a Scene
TLDR
This paper proposes a method that only uses a single RGB image, thus enabling applications in devices or vehicles that do not have LiDAR sensors, and extends the network head by a 3D detection head, which predicts every degree of freedom of a3D bounding box via classification. Expand
PointFPN: A Frustum-based Feature Pyramid Network for 3D Object Detection
TLDR
A frustum-based 3D object detection model named PointFPN, which demonstrates state-of-the-art performance on small object detection on KITTI benchmark and can learn discriminative features which are highly relevant to bounding box parameters at different feature levels. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 44 REFERENCES
2D-Driven 3D Object Detection in RGB-D Images
TLDR
The approach makes best use of the 2D information to quickly reduce the search space in 3D, benefiting from state-of-the-art 2D object detection techniques. Expand
Amodal Detection of 3D Objects: Inferring 3D Bounding Boxes from 2D Ones in RGB-Depth Images
  • Zhuo Deng, L. Latecki
  • Computer Science
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
TLDR
A novel 3D object detection system that simultaneously predicts objects 3D locations, physical sizes, and orientations in indoor scenes is proposed and experiments show the algorithm significantly outperforms the state-of-the-art and indicates 2.5D representation is capable of encoding features for 3D amodal object detection. Expand
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
  • Shuran Song, J. Xiao
  • Computer Science
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
TLDR
This work proposes the first 3D Region Proposal Network (RPN) to learn objectness from geometric shapes and the first joint Object Recognition Network (ORN) to extract geometric features in 3D and color features in 2D. Expand
Sliding Shapes for 3D Object Detection in Depth Images
TLDR
This paper proposes to use depth maps for object detection and design a 3D detector to overcome the major difficulties for recognition, namely the variations of texture, illumination, shape, viewpoint, clutter, occlusion, self-occlusion and sensor noises. Expand
Multi-view 3D Object Detection Network for Autonomous Driving
TLDR
This paper proposes Multi-View 3D networks (MV3D), a sensory-fusion framework that takes both LIDAR point cloud and RGB images as input and predicts oriented 3D bounding boxes and designs a deep fusion scheme to combine region-wise features from multiple views and enable interactions between intermediate layers of different paths. Expand
Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks
TLDR
This paper proposes a computationally efficient approach to detecting objects natively in 3D point clouds using convolutional neural networks (CNNs) by leveraging a feature-centric voting scheme to implement novel convolutionan layers which explicitly exploit the sparsity encountered in the input. Expand
Data-driven 3D Voxel Patterns for object category recognition
TLDR
A novel object representation is proposed, 3D Voxel Pattern (3DVP), that jointly encodes the key properties of objects including appearance,3D shape, viewpoint, occlusion and truncation. Expand
3D Object Proposals for Accurate Object Class Detection
TLDR
This method exploits stereo imagery to place proposals in the form of 3D bounding boxes in the context of autonomous driving and outperforms all existing results on all three KITTI object classes. Expand
SUN RGB-D: A RGB-D scene understanding benchmark suite
TLDR
This paper introduces an RGB-D benchmark suite for the goal of advancing the state-of-the-arts in all major scene understanding tasks, and presents a dataset that enables the train data-hungry algorithms for scene-understanding tasks, evaluate them using meaningful 3D metrics, avoid overfitting to a small testing set, and study cross-sensor bias. Expand
3D Bounding Box Estimation Using Deep Learning and Geometry
TLDR
Although conceptually simple, this method outperforms more complex and computationally expensive approaches that leverage semantic segmentation, instance level segmentation and flat ground priors and produces state of the art results for 3D viewpoint estimation on the Pascal 3D+ dataset. Expand
...
1
2
3
4
5
...