A Benchmark for LiDAR-based Panoptic Segmentation based on KITTI

@article{Behley2021ABF,
  title={A Benchmark for LiDAR-based Panoptic Segmentation based on KITTI},
  author={Jens Behley and Andres Milioto and C. Stachniss},
  journal={2021 IEEE International Conference on Robotics and Automation (ICRA)},
  year={2021},
  pages={13596-13603}
}
Panoptic segmentation is the recently introduced task that tackles semantic segmentation and instance segmentation jointly [18]. In this paper, we present an extension of SemanticKITTI [1], a large-scale dataset providing dense point-wise semantic labels for all sequences of the KITTI Odometry Benchmark [10]. This extension enables training and evaluation of LiDAR-based panoptic segmentation. We provide the data and discuss the processing steps needed to enrich a given semantic annotation with… Expand
Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking
TLDR
This paper introduces the large-scale Panoptic nuScenes benchmark dataset that extends the popularnuScenes dataset with pointwise groundtruth annotations for semantic segmentation, panoptic segmentation and panopti tracking tasks and proposes the novel instance-centric PAT metric that addresses the concerns. Expand
A Technical Survey and Evaluation of Traditional Point Cloud Clustering Methods for LiDAR Panoptic Segmentation
  • Yiming Zhao, Xiao Zhang, Xinming Huang
  • Computer Science
  • 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
  • 2021
TLDR
It is argued geometry-based traditional clustering algorithms are worth being considered by showing a state-of-the-art performance among all published end-to-end deep learning solutions on the panoptic segmentation leaderboard of the SemanticKITTI dataset. Expand
Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation
  • Zixiang Zhou, Yang Zhang, H. Foroosh
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2021
TLDR
This paper presents a fast and robust LiDAR point cloud panoptic segmentation framework, referred to as Panoptic-PolarNet, which learns both semantic segmentation and class-agnostic instance clustering in a single inference network using a polar Bird’s Eye View (BEV) representation. Expand
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
TLDR
This work proposes the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm and presents an efficient learnable clustering module, dynamic shifting, which adapts kernel functions on-the-fly for different instances. Expand
SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clustering
TLDR
This work proposes a novel LiDAR-based panoptic system, called SMAC-Seg, which removes the complex proposal network to segment instances, and proposes to use a novel centroid-aware repel loss as an additional term to effectively supervise the network to differentiate each object cluster with its neighbours. Expand
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
  • Xinge Zhu, Hui Zhou, +5 authors Dahua Lin
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2021
TLDR
A new framework for the outdoor LiDAR segmentation is proposed, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties. Expand
Towards 3D LiDAR-based semantic scene understanding of 3D point cloud sequences: The SemanticKITTI Dataset
TLDR
The SemanticKITTI dataset is presented that provides point-wise semantic annotations of Velodyne HDL-64E point clouds of the KITTI Odometry Benchmark and details on the labeling process to efficiently annotate such a vast amount of point clouds are provided. Expand
LiDAR Panoptic Segmentation for Autonomous Driving
TLDR
This work presents a novel, single-stage, and real-time capable panoptic segmentation approach using a shared encoder with a semantic and instance decoder, which allows the approach to use larger output strides than using transpose convolutions leading to substantial savings in computation time. Expand
Benchmarking Open-World LiDAR Instance Segmentation
  • 2021
With the advent of deep learning, supervised learning led to increased 1 performance in object detection and instance segmentation. However, existing 2 methods explicitly assume that complete worldExpand
Panoptic Segmentation: A Review
TLDR
This is the first comprehensive review of existing panoptic segmentation methods to the best of the authors’ knowledge and provides a comparison of the performance of existing solutions to inform the state-of-the-art and identify their limitations and strengths. Expand
...
1
2
...

References

SHOWING 1-10 OF 49 REFERENCES
SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences
TLDR
A large dataset to propel research on laser-based semantic segmentation, which opens the door for the development of more advanced methods, but also provides plentiful data to investigate new research directions. Expand
RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation
TLDR
This paper proposes a novel post-processing algorithm that deals with problems arising from this intermediate representation of range images as an intermediate representation in combination with a Convolutional Neural Network exploiting the rotating LiDAR sensor model. Expand
Panoptic Segmentation
TLDR
A novel panoptic quality (PQ) metric is proposed that captures performance for all classes (stuff and things) in an interpretable and unified manner and is performed a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task. Expand
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes
TLDR
The Mapillary Vistas Dataset is a novel, large-scale street-level image dataset containing 25000 high-resolution images annotated into 66 object categories with additional, instance-specific labels for 37 classes, aiming to significantly further the development of state-of-the-art methods for visual road-scene understanding. Expand
LiDAR Panoptic Segmentation for Autonomous Driving
TLDR
This work presents a novel, single-stage, and real-time capable panoptic segmentation approach using a shared encoder with a semantic and instance decoder, which allows the approach to use larger output strides than using transpose convolutions leading to substantial savings in computation time. Expand
UPSNet: A Unified Panoptic Segmentation Network
TLDR
A parameter-free panoptic head is introduced which solves thepanoptic segmentation via pixel-wise classification and first leverages the logits from the previous two heads and then innovatively expands the representation for enabling prediction of an extra unknown class which helps better resolving the conflicts between semantic and instance segmentation. Expand
PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things
TLDR
PanopticFusion, a novel online volumetric semantic mapping system at the level of stuff and things, is able to densely predict class labels of a background region and individually segment arbitrary foreground objects and outperformed or compared with state-of-the-art offline 3D DNN methods in both semantic and instance segmentation benchmarks. Expand
3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
TLDR
3D-SIS is introduced, a novel neural network architecture for 3D semantic instance segmentation in commodity RGB-D scans that leverages high-resolution RGB input by associating 2D images with the volumetric grid based on the pose alignment of the 3D reconstruction. Expand
SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances
TLDR
Experimental results show that SemanticPOSS can help to improve the prediction accuracy of dynamic objects as people, car in some degree and the data is collected in Peking University and uses the same data format as SemanticKITTI. Expand
The Cityscapes Dataset for Semantic Urban Scene Understanding
TLDR
This work introduces Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling, and exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Expand
...
1
2
3
4
5
...