Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping

@article{Lin2019UsingSD,
  title={Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping},
  author={Yunzhi Lin and Chao Tang and Fu-Jen Chu and Patricio A. Vela},
  journal={2020 IEEE International Conference on Robotics and Automation (ICRA)},
  year={2019},
  pages={10494-10501}
}
  • Yunzhi LinChao Tang P. Vela
  • Published 12 September 2019
  • Computer Science
  • 2020 IEEE International Conference on Robotics and Automation (ICRA)
A segmentation-based architecture is proposed to decompose objects into multiple primitive shapes from monocular depth input for robotic manipulation. The backbone deep network is trained on synthetic data with 6 classes of primitive shapes generated by a simulation engine. Each primitive shape is designed with parametrized grasp families, permitting the pipeline to identify multiple grasp candidates per shape primitive region. The grasps are priority ordered via proposed ranking algorithm… 
View on IEEE
arxiv.org
16 Citations
Background Citations
8
Methods Citations
5

Figures and Tables from this paper

16 Citations

Primitive Shape Recognition for Object Grasping

The outcomes support the hypothesis that explicitly encoding shape primitives within a grasping pipeline should boost grasping performance, including task-free and task-relevant grasp prediction.

GKNet: Grasp keypoint network for grasp candidates detection

GKNet outperforms reference baselines in static and dynamic grasping experiments while showing robustness to varied camera viewpoints and moderate clutter, and confirms the hypothesis that grasp keypoints are an effective output representation for deep grasp networks that provide robusts to expected nuisance factors.

Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis

An eye-in-hand learning-based approach for hand pre-shape classification from RGB sequences and shows that models trained with the synthetic dataset achieve better generalization performance than models trained on real data.

Custom Grasping: A Region-Based Robotic Grasping Detection Method in Industrial Cyber-Physical Systems

A region-based few shot learning approach that enables the robot to detect the best grasping point-pair autonomously and quickly and is feasible in robot automation for handling a class of objects.

Recognizing Object Affordances to Support Scene Reasoning for Manipulation Tasks.

An affordance recognition pipeline based on a category-agnostic region proposal network for proposing instance regions of an image across categories and a self-attention mechanism trained to interpret each proposal learns to capture rich contextual dependencies through the region.

Scalable, physics-aware 6D pose estimation for robot manipulation

Algorithms to generate and validate object poses online based on the objects’ geometry andbased on the physical consistency of their scene-level interactions are proposed, providing robustness even when there exists a domain gap between the synthetic training and the real test scenarios.

Learning-based Fast Path Planning in Complex Environments

A novel path planning algorithm to achieve fast path planning in complex environments by using a learning-based prediction module and a sampling-based path planning module that can achieve much better performance in terms of planning time, success rate, and path length.

Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping

A novel framework of vision-guided tactile poking for transparent objects grasping is proposed, which could be adopted by other force or tactile sensors and could be used for grasping of other challenging objects.

Active Scene Understanding from Image Sequences for Next-Generation Computer Vision

This work creates a prototype computer vision system for acquiring a deeper scene understanding and shows that this approach is capable of solving many common shortcomings of traditional object recognition approaches including the understanding of 3D occlusion, the ability to aggregate information over many frames, theUnderstanding of object permanence or the active control of the camera in a beneficial way.

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

This comprehensive review of deep reinforcement learning in the manipulation field may be valuable for researchers and practitioners because they can expedite the establishment of important guidelines.

References

SHOWING 1-10 OF 47 REFERENCES

Domain Randomization and Generative Models for Robotic Grasping

A novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis and can achieve a >90% success rate on previously unseen realistic objects at test time in simulation despite having only been trained on random objects.

Real-World Multiobject, Multigrasp Detection

A deep learning architecture is proposed to predict graspable locations for robotic manipulation by defining the learning problem to be classified with null hypothesis competition instead of regression, the deep neural network with red, green, blue and depth image input predicts multiple grasp candidates for a single object or multiple objects, in a single shot.

Selection of robot pre-grasps using box-based shape approximation

  • K. HuebnerD. Kragic
  • Computer Science
    2008 IEEE/RSJ International Conference on Intelligent Robots and Systems
  • 2008
It is motivated how boxes as one of the simplest representations can be applied in a more sophisticated manner to generate task-dependent grasps.

Grasp planning for everyday objects based on primitive shape representation for parallel jaw grippers

  • N. YamanobeK. Nagata
  • Computer Science
    2010 IEEE International Conference on Robotics and Biomimetics
  • 2010
Seven kinds of shape primitives utilized for abstracting objects to be grasped are proposed for efficient grasp planning and an experimental result of the application of this shape primitive based grasp planning method to a mobile manipulator is shown.

Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network

This work is the first to directly plan high quality multifingered grasps in configuration space using a deep neural network without the need of an external planner and shows that the planning method outperforms existing planning methods for neural networks.

A New Approach Based on Two-stream CNNs for Novel Objects Grasping in Clutter

A deep learning approach is applied to solve the problem about grasping novel objects in clutter by proposing a ‘grasp circle’ method to find more potential grasps in each sampling point with less cost, which is parameterized by the size of the gripper.

On-Policy Dataset Synthesis for Learning Robot Grasping Policies Using Fully Convolutional Deep Networks

A synthetic data sampling distribution is proposed that combines grasps sampled from the policy action set with guiding samples from a robust grasping supervisor that has full state knowledge to improve rate and reliability of the learned robot policy.

Data-Driven Grasp Synthesis—A Survey

A review of the work on data-driven grasp synthesis and the methodologies for sampling and ranking candidate grasps and an overview of the different methodologies are provided, which draw a parallel to the classical approaches that rely on analytic formulations.

High precision grasp pose detection in dense clutter

This paper proposes two new representations of grasp candidates, and quantifies the effect of using prior knowledge of two forms: instance or category knowledge of the object to be grasped, and pretraining the network on simulated depth data obtained from idealized CAD models.

Automatic grasp planning using shape primitives

This paper aims to simplify automatic grasp planning for robotic hands by modeling an object as a set of shape primitives, such as spheres, cylinders, cones and boxes, to generate aSet of grasp starting positions and pregrasp shapes that can then be tested on the object model.