Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation

  title={Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation},
  author={Andrew Price and Kun Huang and Dmitry Berenson},
  journal={2021 IEEE International Conference on Robotics and Automation (ICRA)},
Despite rapid progress in scene segmentation in recent years, 3D segmentation methods are still limited when there is severe occlusion. The key challenge is estimating the segment boundaries of (partially) occluded objects, which are inherently ambiguous when considering only a single frame. In this work, we propose Multihypothesis Segmentation Tracking (MST), a novel method for volumetric segmentation in changing scenes, which allows scene ambiguity to be tracked and our estimates to be… 

Figures from this paper



Tracking-based interactive segmentation of textureless objects

This paper describes a textureless object segmentation approach for autonomous service robots acting in human living environments. The proposed system allows a robot to effectively segment

MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM

This system is the first system to generate an object-level dynamic volumetric map from a single RGB-D camera, which can be used directly for robotic tasks and demonstrates its effectiveness by quantitatively and qualitatively testing it on both synthetic and real-world sequences.

Fast Online Object Tracking and Segmentation: A Unifying Approach

This method improves the offline training procedure of popular fully-convolutional Siamese approaches for object tracking by augmenting their loss with a binary segmentation task, and operates online, producing class-agnostic object segmentation masks and rotated bounding boxes at 55 frames per second.

SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes

sceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training.

RGB-D object detection and semantic segmentation for autonomous manipulation in clutter

The experiments show that object detection and semantic segmentation complement each other and can be combined to yield reliable object perception.

Tree-Cut for Probabilistic Image Segmentation

This paper presents a new probabilistic generative model for image segmentation, i.e. the task of partitioning an image into homogeneous regions. Our model is grounded on a mid-level image

3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans

3D-SIS is introduced, a novel neural network architecture for 3D semantic instance segmentation in commodity RGB-D scans that leverages high-resolution RGB input by associating 2D images with the volumetric grid based on the pose alignment of the 3D reconstruction.

Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks

Convolutional Oriented Boundaries gives a significant leap in performance over the state-of-the-art, and generalizes very well to unseen categories and datasets, and learning to estimate not only contour strength but also orientation provides more accurate results.

SOLO: Segmenting Objects by Locations

A new, embarrassingly simple approach to instance segmentation in images by introducing the notion of "instance categories", which assigns categories to each pixel within an instance according to the instance's location and size thus nicely converting instance mask segmentation into a classification-solvable problem.

Stochastic Segmentation Trees for Multiple Ground Truths

A tree-structured probabilistic model, the stochastic segmentation tree, is formulated that represents a distribution over segmentations of a given image and is able to successfully capture the range of ground truths and to produce novel plausible segmentations beyond those found in the data.