SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes

@article{Pham2018SceneCutJG,
  title={SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes},
  author={Trung T. Pham and Thanh-Toan Do and Niko S{\"u}nderhauf and Ian D. Reid},
  journal={2018 IEEE International Conference on Robotics and Automation (ICRA)},
  year={2018},
  pages={1-9}
}
This paper presents SceneCut, a novel approach to jointly discover previously unseen objects and non-object surfaces using a single RGB-D image. SceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training. SceneCut automatically decomposes a scene into meaningful regions which… 
View on IEEE
arxiv.org
30 Citations
Background Citations
10
Methods Citations
6
Results Citations
3

Figures and Tables from this paper

30 Citations

Citation Type

Citation Type

Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery
TLDR
This work presents an approach to incrementally build volumetric object-centric maps during online scanning with a localized RGB-D camera and demonstrates that the proposed approach for building instance-level semantic maps is competitive with state-of-the-art methods, while additionally able to discover objects of unseen categories.
Efficient point cloud segmentation approach using energy optimization with geometric features for 3D scene understanding.
TLDR
Experiments on the Object Cluttered Indoor Dataset dataset indicate that the proposed method can outperform the representative segmentation algorithms in terms of weighted overlap and accuracy, while the method has good robustness and real-time performance.
Unknown Object Segmentation from Stereo Images
TLDR
This work proposes a novel object instance segmentation approach that does not require any semantic or geometric information of the objects beforehand before it is computed, and uses the original image pair to infer the object instances and a dense depth map.
Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling
TLDR
A Hierarchical Occlusion Modeling (HOM) scheme designed to reason about the occlusion by assigning a hierarchy to a feature fusion and prediction order is proposed.
Geometric property-based convolutional neural network for indoor object detection
TLDR
This article focuses on region-based convolutional neural network (CNN) detector and proposes a geometric property-based Faster R-CNN method (GP-Faster) for indoor object detection, which incorporates geometric property in FasterR-CNN to improve the detection performance.
Unsupervised Foreground Extraction via Deep Region Competition
TLDR
This work rethink the foreground extraction by reconciling energy-based prior with generative image modeling in the form of Mixture of Experts (MoE), where learned pixel re-assignment is introduced as the essential inductive bias to capture the regularities of background regions.
Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation
TLDR
This work proposes a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data by utilizing a metric learning loss function and demonstrates that non-photorealistic synthetic RGB and depth images can be used to learn feature embeddeddings that transfer well to real-world images.
EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasets
TLDR
This paper presents the EasyLabel tool for easily acquiring high-quality ground truth annotation of objects at pixel-level in densely cluttered scenes and reveals the usefulness of EasyLabel and OCID to better understand the challenges that robots face in the real world.
Unseen Object Instance Segmentation for Robotic Environments
TLDR
This work shows that the proposed UOIS-Net method can produce sharp and accurate segmentation masks, outperforming state-of-the-art methods on unseen object instance segmentation, and shows that it can segment unseen objects for robot grasping.
Inferring Occluded Geometry Improves Performance when Retrieving an Object from Dense Clutter
TLDR
This work augments a manipulation planner for cluttered environments with a state-of-the-art deep neural network for shape completion as well as a volumetric memory system, allowing the robot to reason about what may be contained in occluded areas.
...
...

References

SHOWING 1-10 OF 28 REFERENCES
Decomposing a scene into geometric and semantically consistent regions
TLDR
A region-based model which combines appearance and scene geometry to automatically decompose a scene into semantically meaningful regions and which achieves state-of-the-art performance on the tasks of both multi-class image segmentation and geometric reasoning.
Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
TLDR
This work proposes algorithms for object boundary detection and hierarchical segmentation that generalize the gPb-ucm approach of [2] by making effective use of depth information and shows how this contextual information in turn improves object recognition.
Indoor Segmentation and Support Inference from RGBD Images
TLDR
The goal is to parse typical, often messy, indoor scenes into floor, walls, supporting surfaces, and object regions, and to recover support relationships, to better understand how 3D cues can best inform a structured 3D interpretation.
Object discovery in 3D scenes via shape analysis
We present a method for discovering object models from 3D meshes of indoor environments. Our algorithm first decomposes the scene into a set of candidate mesh segments and then ranks each segment…
Object Partitioning Using Local Convexity
TLDR
A new, efficient learning- and model-free approach for the segmentation of 3D point clouds into object parts that is comparable to state-of-the-art methods which incorporate high-level concepts involving classification, learning and model fitting.
Geometrically consistent plane extraction for dense indoor 3D maps segmentation
TLDR
This paper presents an unsupervised geometric-based approach for the segmentation of 3D point clouds into objects and meaningful scene structures and proposes a novel global plane extraction algorithm for robustly discovering the underlying planes in the scene.
Hierarchical Higher-Order Regression Forest Fields: An Application to 3D Indoor Scene Labelling
TLDR
This paper addresses the problem of semantic segmentation of 3D indoor scenes reconstructed from RGB-D images by relaxing the labelling problem to a regression, and generalizing the higher-order associative P n Potts model to a new family of arbitrary higher- order models based on regression forests.
Meaningful maps with object-oriented semantic mapping
TLDR
This paper simultaneously build geometric point cloud models of previously unseen instances of known object classes and create a map that contains these object models as central entities that leverages sparse, feature-based RGB-D SLAM, image-based deep-learning object detection and 3D unsupervised segmentation.
Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks
TLDR
Convolutional Oriented Boundaries gives a significant leap in performance over the state-of-the-art, and generalizes very well to unseen categories and datasets, and learning to estimate not only contour strength but also orientation provides more accurate results.
Mask R-CNN
TLDR
This work presents a conceptually simple, flexible, and general framework for object instance segmentation that outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners.
...
...