Unseen Object Instance Segmentation for Robotic Environments

@article{Xie2021UnseenOI,
  title={Unseen Object Instance Segmentation for Robotic Environments},
  author={Christopher Xie and Yu Xiang and Arsalan Mousavian and Dieter Fox},
  journal={IEEE Transactions on Robotics},
  year={2021},
  volume={37},
  pages={1343-1359}
}
In order to function in unstructured environments, robots need the ability to recognize unseen objects. We take a step in this direction by tackling the problem of segmenting unseen object instances in tabletop environments. However, the type of large-scale real-world dataset required for this task typically does not exist for most robotic settings, which motivates the use of synthetic data. Our proposed method, unseen object instance segmentation (UOIS)-Net, separately leverages synthetic RGB… 
Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling
TLDR
A Hierarchical Occlusion Modeling (HOM) scheme designed to reason about the occlusion by assigning a hierarchy to a feature fusion and prediction order is proposed.
Unknown Object Segmentation from Stereo Images
TLDR
This work proposes a novel object instance segmentation approach that does not require any semantic or geometric information of the objects beforehand before it is computed, and uses the original image pair to infer the object instances and a dense depth map.
SIMstack: A Generative Shape and Instance Model for Unordered Object Stacks
TLDR
SIMstack, a depthconditioned Variational Auto-Encoder trained on a dataset of objects stacked under physics simulation, is proposed and an application for precise (non-disruptive) object grasping of unknown objects from a single depth view is demonstrated.
RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks
TLDR
A novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks is proposed, and an application is demonstrated that uses uncertainty estimates generated by the method to guide a manipulator, leading to efficient understanding of cluttered scenes.
ICM-3D: Instantiated Category Modeling for 3D Instance Segmentation
TLDR
This paper proposes ICM-3D, a single-step method to segment 3D instances via instantiated categorization and shows that it obtains inspiring performance across multiple frameworks, backbones and benchmarks.
LRGNet: Learnable Region Growing for Class-Agnostic Point Cloud Segmentation
TLDR
The proposed method is able to segment any class of objects using a single deep neural network without any assumptions about their shapes and sizes, and outperforms competing methods on 6 different evaluation metrics.
Depth-aware Object Segmentation and Grasp Detection for Robotic Picking Tasks
In this paper, we present a novel deep neural network architecture for joint classagnostic object segmentation and grasp detection for robotic picking tasks using a parallelplate gripper. We
CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation
TLDR
A novel, object-centric canonical representation at the category level is proposed, which allows establishing dense correspondence across object instances and transferring task-relevant grasps to novel instances.
Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordances
TLDR
It is shown that following this strategy a task-and-motion planner can be used to plan intelligent behaviors even in the absence of a priori knowledge regarding the set of manipulable objects, their geometries, and their affordances.
InSeGAN: A Generative Approach to Segmenting Identical Instances in Depth Images
TLDR
This paper designs a novel GAN architecture to synthesize a multiple-instance depth image with independent control over each instance, and proposes an instance pose encoder that learns to take in a generated depth image and reproduce the pose code vectors for all of the object instances.
...
1
2
...

References

SHOWING 1-10 OF 88 REFERENCES
The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
TLDR
A novel method is proposed that separately leverages synthetic RGB and synthetic depth for unseen object instance segmentation, and is able to learn from synthetic RGB-D data where the RGB is non-photorealistic.
ClusterNet: Instance Segmentation in RGB-D Images
TLDR
This model proposes a method for instance-level segmentation that uses RGB-D data as input and provides detailed information about the location, geometry and number of individual objects in the scene and outperforms the state-of-the-art instance segmentation method on the synthesized dataset.
ClusterNet: 3D Instance Segmentation in RGB-D Images
TLDR
This work proposes a method for instance-level segmentation that uses RGB-D data as input and provides detailed information about the location, geometry and number of individual objects in the scene and outperforms the stateof-the-art instance segmentation method on the synthesized dataset.
OccuSeg: Occupancy-Aware 3D Instance Segmentation
TLDR
This paper defines “3D occupancy size”, as the number of voxels occupied by each instance, and OccuSeg, an occupancy-aware 3D instance segmentation scheme is proposed, which achieves state-of-theart performance on 3 real-world datasets, i.e. ScanNetV2, S3DIS and SceneNN, while maintaining high efficiency.
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
TLDR
This network is the first deep network trained only on synthetic data that is able to achieve state-of-the-art performance on 6-DoF object pose estimation and demonstrates a real-time system estimating object poses with sufficient accuracy for real-world semantic grasping of known household objects in clutter by a real robot.
Towards Segmenting Anything That Moves
TLDR
This work proposes a simple learning-based approach for spatio-temporal grouping that leverages motion cues from optical flow as a bottom-up signal for separating objects from each other, and shows that this model matches top-down methods on common categories, while significantly out-performing both top- down and bottom- up methods on never-before-seen categories.
Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data
TLDR
A method for automated dataset generation is presented and a variant of Mask R-CNN is trained with domain randomization on the generated dataset to perform category-agnostic instance segmentation without any hand-labeled data and the model is deployed in an instance-specific grasping pipeline to demonstrate its usefulness in a robotics application.
Segmentation of unknown objects in indoor environments
TLDR
This work presents a framework for segmenting unknown objects in RGB-D images suitable for robotics tasks such as object search, grasping and manipulation and shows evaluation of the relations and results on a database of different test sets, demonstrating that the approach can segment objects of various shapes in cluttered table top scenes.
SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes
TLDR
sceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training.
Microsoft COCO: Common Objects in Context
We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene
...
1
2
3
4
5
...