Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping
@article{Lin2019UsingSD, title={Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping}, author={Yunzhi Lin and Chao Tang and Fu-Jen Chu and Patricio A. Vela}, journal={2020 IEEE International Conference on Robotics and Automation (ICRA)}, year={2019}, pages={10494-10501} }
A segmentation-based architecture is proposed to decompose objects into multiple primitive shapes from monocular depth input for robotic manipulation. The backbone deep network is trained on synthetic data with 6 classes of primitive shapes generated by a simulation engine. Each primitive shape is designed with parametrized grasp families, permitting the pipeline to identify multiple grasp candidates per shape primitive region. The grasps are priority ordered via proposed ranking algorithm…
Figures and Tables from this paper
16 Citations
Primitive Shape Recognition for Object Grasping
- Computer ScienceArXiv
- 2022
The outcomes support the hypothesis that explicitly encoding shape primitives within a grasping pipeline should boost grasping performance, including task-free and task-relevant grasp prediction.
GKNet: Grasp keypoint network for grasp candidates detection
- Computer ScienceInt. J. Robotics Res.
- 2022
GKNet outperforms reference baselines in static and dynamic grasping experiments while showing robustness to varied camera viewpoints and moderate clutter, and confirms the hypothesis that grasp keypoints are an effective output representation for deep grasp networks that provide robusts to expected nuisance factors.
Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis
- Computer Science2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- 2022
An eye-in-hand learning-based approach for hand pre-shape classification from RGB sequences and shows that models trained with the synthetic dataset achieve better generalization performance than models trained on real data.
Custom Grasping: A Region-Based Robotic Grasping Detection Method in Industrial Cyber-Physical Systems
- Computer ScienceIEEE Transactions on Automation Science and Engineering
- 2023
A region-based few shot learning approach that enables the robot to detect the best grasping point-pair autonomously and quickly and is feasible in robot automation for handling a class of objects.
Recognizing Object Affordances to Support Scene Reasoning for Manipulation Tasks.
- Computer Science, Psychology
- 2020
An affordance recognition pipeline based on a category-agnostic region proposal network for proposing instance regions of an image across categories and a self-attention mechanism trained to interpret each proposal learns to capture rich contextual dependencies through the region.
Scalable, physics-aware 6D pose estimation for robot manipulation
- Computer Science
- 2020
Algorithms to generate and validate object poses online based on the objects’ geometry andbased on the physical consistency of their scene-level interactions are proposed, providing robustness even when there exists a domain gap between the synthetic training and the real test scenarios.
Learning-based Fast Path Planning in Complex Environments
- Computer Science2021 IEEE International Conference on Robotics and Biomimetics (ROBIO)
- 2021
A novel path planning algorithm to achieve fast path planning in complex environments by using a learning-based prediction module and a sampling-based path planning module that can achieve much better performance in terms of planning time, success rate, and path length.
Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping
- Computer ScienceIEEE/ASME Transactions on Mechatronics
- 2022
A novel framework of vision-guided tactile poking for transparent objects grasping is proposed, which could be adopted by other force or tactile sensors and could be used for grasping of other challenging objects.
Active Scene Understanding from Image Sequences for Next-Generation Computer Vision
- Computer Science
- 2020
This work creates a prototype computer vision system for acquiring a deeper scene understanding and shows that this approach is capable of solving many common shortcomings of traditional object recognition approaches including the understanding of 3D occlusion, the ability to aggregate information over many frames, theUnderstanding of object permanence or the active control of the camera in a beneficial way.
Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations
- Computer ScienceIEEE Access
- 2020
This comprehensive review of deep reinforcement learning in the manipulation field may be valuable for researchers and practitioners because they can expedite the establishment of important guidelines.
References
SHOWING 1-10 OF 47 REFERENCES
Domain Randomization and Generative Models for Robotic Grasping
- Computer Science2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- 2018
A novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis and can achieve a >90% success rate on previously unseen realistic objects at test time in simulation despite having only been trained on random objects.
Real-World Multiobject, Multigrasp Detection
- Computer ScienceIEEE Robotics and Automation Letters
- 2018
A deep learning architecture is proposed to predict graspable locations for robotic manipulation by defining the learning problem to be classified with null hypothesis competition instead of regression, the deep neural network with red, green, blue and depth image input predicts multiple grasp candidates for a single object or multiple objects, in a single shot.
Selection of robot pre-grasps using box-based shape approximation
- Computer Science2008 IEEE/RSJ International Conference on Intelligent Robots and Systems
- 2008
It is motivated how boxes as one of the simplest representations can be applied in a more sophisticated manner to generate task-dependent grasps.
Grasp planning for everyday objects based on primitive shape representation for parallel jaw grippers
- Computer Science2010 IEEE International Conference on Robotics and Biomimetics
- 2010
Seven kinds of shape primitives utilized for abstracting objects to be grasped are proposed for efficient grasp planning and an experimental result of the application of this shape primitive based grasp planning method to a mobile manipulator is shown.
Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network
- Computer ScienceISRR
- 2017
This work is the first to directly plan high quality multifingered grasps in configuration space using a deep neural network without the need of an external planner and shows that the planning method outperforms existing planning methods for neural networks.
A New Approach Based on Two-stream CNNs for Novel Objects Grasping in Clutter
- Computer ScienceJ. Intell. Robotic Syst.
- 2019
A deep learning approach is applied to solve the problem about grasping novel objects in clutter by proposing a ‘grasp circle’ method to find more potential grasps in each sampling point with less cost, which is parameterized by the size of the gripper.
On-Policy Dataset Synthesis for Learning Robot Grasping Policies Using Fully Convolutional Deep Networks
- Computer ScienceIEEE Robotics and Automation Letters
- 2019
A synthetic data sampling distribution is proposed that combines grasps sampled from the policy action set with guiding samples from a robust grasping supervisor that has full state knowledge to improve rate and reliability of the learned robot policy.
Data-Driven Grasp Synthesis—A Survey
- Computer ScienceIEEE Transactions on Robotics
- 2014
A review of the work on data-driven grasp synthesis and the methodologies for sampling and ranking candidate grasps and an overview of the different methodologies are provided, which draw a parallel to the classical approaches that rely on analytic formulations.
High precision grasp pose detection in dense clutter
- Computer Science2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- 2016
This paper proposes two new representations of grasp candidates, and quantifies the effect of using prior knowledge of two forms: instance or category knowledge of the object to be grasped, and pretraining the network on simulated depth data obtained from idealized CAD models.
Automatic grasp planning using shape primitives
- Computer Science2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422)
- 2003
This paper aims to simplify automatic grasp planning for robotic hands by modeling an object as a set of shape primitives, such as spheres, cylinders, cones and boxes, to generate aSet of grasp starting positions and pregrasp shapes that can then be tested on the object model.