Learning to Segment 3D Point Clouds in 2D Image Space

@article{Lyu2020LearningTS,
  title={Learning to Segment 3D Point Clouds in 2D Image Space},
  author={Yecheng Lyu and X. Huang and Ziming Zhang},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={12252-12261}
}
In contrast to the literature where local patterns in 3D point clouds are captured by customized convolutional operators, in this paper we study the problem of how to effectively and efficiently project such point clouds into a 2D image space so that traditional 2D convolutional neural networks (CNNs) such as U-Net can be applied for segmentation. To this end, we are motivated by graph drawing and reformulate it as an integer programming problem to learn the topology-preserving graph-to-grid… Expand
Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
TLDR
This work develops a 3D cylinder partition and a3D cylinder convolution based framework, termed as Cylinder3D, which exploits the 3D topology relations and structures of driving-scene point clouds and introduces a dimension-decomposition based context modeling module to explore the high-rank context information in point clouds in a progressive manner. Expand
Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
TLDR
This paper presents an urban-scale photogrammetric point cloud dataset with nearly three billion richly annotated points, which is five times the number of labeled points than the existing largest point cloud datasets. Expand
A voxelized point clouds representation for object classification and segmentation on 3D data
TLDR
This work builds a sampling module based on a hybrid model that combines point cloud and voxel data and uses a point quantization method to ensure that each cell has the same number of points, making it easier for 3D convolution kernels to extract object features. Expand
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
TLDR
A new framework for the outdoor LiDAR segmentation is proposed, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pat-tern while maintaining these inherent properties. Expand
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
  • Xinge Zhu, Hui Zhou, +6 authors Dahua Lin
  • Medicine, Computer Science
  • IEEE transactions on pattern analysis and machine intelligence
  • 2021
TLDR
A new framework for the outdoor LiDAR segmentation is proposed, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties. Expand
EllipsoidNet: Ellipsoid Representation for Point Cloud Classification and Segmentation
TLDR
This paper proposes a novel 2D representation method that projects a point cloud onto an ellipsoid surface space, where local patterns are well exposed in ellIPsoid-level and point-level, and a novel convolutional neural network named EllipsoIDNet is proposed to utilize those features for point cloud classification and segmentation applications. Expand
Exploiting Local Geometry for Feature and Graph Construction for Better 3D Point Cloud Processing with Graph Neural Networks
TLDR
It is demonstrated with multiple challenging benchmarks, with relatively clean CAD models, as well as with real world noisy scans, that the proposed method achieves state of the art results on benchmarks for 3D classification, part segmentation and semantic segmentation. Expand
PnP-3D: A Plug-and-Play for 3D Point Clouds
With the help of the deep learning paradigm, many point cloud networks have been invented for visual analysis. However, there is great potential for development of these networks since the givenExpand
Revisiting 2D Convolutional Neural Networks for Graph-based Applications
  • Yecheng Lyu, Xinming Huang, Ziming Zhang
  • Computer Science, Medicine
  • IEEE transactions on pattern analysis and machine intelligence
  • 2021
TLDR
This paper proposes two novel graph-to-grid mapping schemes, namely, graph-preserving grid layout and its extension Hierarchical GPGL for computational efficiency and proposes a novel vertex separation penalty that encourages graph vertices to lay on the grid without any overlap. Expand
Theoretical bounds on data requirements for the ray-based classification
TLDR
A bound on the number of rays necessary for shape classification, defined by key angular metrics, for arbitrary convex shapes is established and enables a different approach for estimating high-dimensional shapes using substantially fewer data elements than volumetric or surface-based approaches. Expand
...
1
2
...

References

SHOWING 1-10 OF 81 REFERENCES
RGCNN: Regularized Graph CNN for Point Cloud Segmentation
TLDR
A regularized graph convolutional neural network (RGCNN) that directly consumes point clouds is proposed that significantly reduces the computational complexity while achieving competitive performance with the state of the art. Expand
A-CNN: Annularly Convolutional Neural Networks on Point Clouds
TLDR
A new method to define and compute convolution directly on 3D point clouds by the proposed annular convolution that can better capture the local neighborhood geometry of each point by specifying the (regular and dilated) ring-shaped structures and directions in the computation. Expand
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
TLDR
This paper designs a novel type of neural network that directly consumes point clouds, which well respects the permutation invariance of points in the input and provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Expand
PointConv: Deep Convolutional Networks on 3D Point Clouds
TLDR
The dynamic filter is extended to a new convolution operation, named PointConv, which can be applied on point clouds to build deep convolutional networks and is able to achieve state-of-the-art on challenging semantic segmentation benchmarks on 3D point clouds. Expand
Interpolated Convolutional Networks for 3D Point Cloud Understanding
TLDR
A novel Interpolated Convolution operation, InterpConv, is proposed to tackle the point cloud feature learning and understanding problem, to utilize a set of discrete kernel weights and interpolate point features to neighboring kernel-weight coordinates by an interpolation function for convolution. Expand
Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs
TLDR
It is argued that the organization of 3D point clouds can be efficiently captured by a structure called superpoint graph (SPG), derived from a partition of the scanned scene into geometrically homogeneous elements. Expand
VV-Net: Voxel VAE Net With Group Convolutions for Point Cloud Segmentation
We present a novel algorithm for point cloud segmentation.Our approach transforms unstructured point clouds into regular voxel grids, and further uses a kernel-based interpolated variationalExpand
Dynamic Graph CNN for Learning on Point Clouds
TLDR
This work proposes a new neural network module suitable for CNN-based high-level tasks on point clouds, including classification and segmentation called EdgeConv, which acts on graphs dynamically computed in each layer of the network. Expand
Recurrent Slice Networks for 3D Segmentation of Point Clouds
TLDR
This work presents a novel 3D segmentation framework, RSNet1, to efficiently model local structures in point clouds using a combination of a novel slice pooling layer, Recurrent Neural Network layers, and a slice unpooling layer. Expand
SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud
TLDR
This work introduces a new model SqueezeSegV2, which is more robust against dropout noises in LiDAR point cloud and therefore achieves significant accuracy improvement, and a domain-adaptation training pipeline consisting of three major components: learned intensity rendering, geodesic correlation alignment, and progressive domain calibration. Expand
...
1
2
3
4
5
...