Corpus ID: 198147345

Inferring Occluded Geometry Improves Performance when Retrieving an Object from Dense Clutter

  title={Inferring Occluded Geometry Improves Performance when Retrieving an Object from Dense Clutter},
  author={Andrew Price and Linyi Jin and Dmitry Berenson},
Object search -- the problem of finding a target object in a cluttered scene -- is essential to solve for many robotics applications in warehouse and household environments. However, cluttered environments entail that objects often occlude one another, making it difficult to segment objects and infer their shapes and properties. Instead of relying on the availability of CAD or other explicit models of scene objects, we augment a manipulation planner for cluttered environments with a state-of… Expand
Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling
A Hierarchical Occlusion Modeling (HOM) scheme designed to reason about the occlusion by assigning a hierarchy to a feature fusion and prediction order is proposed. Expand
Task-Driven Perception and Manipulation for Constrained Placement of Unknown Objects
This work deals with pick-and-constrained placement of objects without access to geometric models, and proposes an algorithmic framework, which performs manipulation planning simultaneously over a conservative and an optimistic estimate of the object’s volume. Expand
kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completion
A new hybrid object representation consisting of semantic keypoint and dense geometry (a point cloud or mesh) as the interface between the perception module and motion planner is proposed, which encodes both the object target configuration and physical feasibility for a category of objects. Expand
Task-driven Perception and Manipulation for Constrained Placement with No Shape Priors
Recent progress in robotic manipulation has dealt with the case of no prior object models in the context of relatively simple tasks, such as bin-picking. Existing methods for more constrainedExpand
Robotic Grasping of Fully-Occluded Objects using RF Perception
RF-Grasp introduces two main innovations: an RF-visual servoing controller that uses the RFID’s location to selectively explore the environment and plan an efficient trajectory toward an occluded target, and an RFvisual deep reinforcement learning network that can learn and execute efficient, complex policies for decluttering and grasping. Expand
Associative3D: Volumetric Reconstruction from Sparse Views
A new approach is proposed that estimates reconstructions, distributions over the camera/object and camera/camera transformations, as well as an inter-view object affinity matrix, and is jointly reasoned over to produce the most likely explanation of the scene. Expand
Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation
Multihypothesis Segmentation Tracking (MST) is proposed, a novel method for volumetric segmentation in changing scenes, which allows scene ambiguity to be tracked and their estimates to be adjusted over time as the authors interact with the scene. Expand
Efficient Unicontact Grasping in Cluttered Scenes
Efficient Unicontact Grasping in Cluttered Scenes
X-Ray: Mechanical Search for an Occluded Object by Minimizing Support of Learned Occupancy Distributions
X-Ray, an algorithm based on learned occupancy distributions that minimizes support of the learned distribution as part of a mechanical search policy in both simulated and real environments, is introduced. Expand


Manipulation-based active search for occluded objects
A novel generative model for representing container contents by using object co-occurrence information and spatial constraints is presented, which is used to guide an agent to explore containers where the target is likely, potentially needing to move occluding objects to enable further perception. Expand
Act to See and See to Act: POMDP planning for objects search in clutter
  • Jueya Li, David Hsu, Wee Sun Lee
  • Engineering, Computer Science
  • 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2016
This work model the problem of objects search in clutter as a Partially Observable Markov Decision Process (POMDP), formulating it as a problem of optimal decision making under uncertainty, and is able to adapt online POMDP planners to handle objects search problems with large state space and action space. Expand
Learning to Manipulate Unknown Objects in Clutter by Reinforcement
A fully autonomous robotic system for grasping objects in dense clutter is presented, which learns online, from scratch, to manipulate the objects by trial and error using stochastic transitions between the observed states, using nonparametric density estimation. Expand
SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes
sceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training. Expand
Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching
A robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments and the key new feature is that it can recognize novel objects that are not known to the system. Expand
Structured Prediction of Unobserved Voxels from a Single Depth Image
This work proposes an algorithm that can complete the unobserved geometry of tabletop-sized objects, based on a supervised model trained on already available volumetric elements, that maps from a local observation in a single depth image to an estimate of the surface shape in the surrounding neighborhood. Expand
Using Manipulation Primitives for Object Sorting in Cluttered Environments
A robust pipeline that combines perception and manipulation to accurately sort objects by some property is presented, which results in the ability to sort cluttered object piles accurately. Expand
Fast object localization and pose estimation in heavy clutter for robotic bin picking
We present a practical vision-based robotic bin-picking system that performs detection and three-dimensional pose estimation of objects in an unstructured bin using a novel camera design, picks upExpand
Object search by manipulation
This work contributes a formulation of the object search by manipulation problem using visibility and accessibility relations between objects, and proposes a greedy algorithm that is optimal under certain conditions and an implementation of both algorithms on a real robot using a real object detection system. Expand
Indoor Segmentation and Support Inference from RGBD Images
The goal is to parse typical, often messy, indoor scenes into floor, walls, supporting surfaces, and object regions, and to recover support relationships, to better understand how 3D cues can best inform a structured 3D interpretation. Expand