Learning to Singulate Objects using a Push Proposal Network

@article{Eitel2017LearningTS,
  title={Learning to Singulate Objects using a Push Proposal Network},
  author={Andreas Eitel and Nico Hauff and Wolfram Burgard},
  journal={ArXiv},
  year={2017},
  volume={abs/1707.08101}
}
Learning to act in unstructured environments, such as cluttered piles of objects, poses a substantial challenge for manipulation robots. We present a novel neural network-based approach that separates unknown objects in clutter by selecting favourable push actions. Our network is trained from data collected through autonomous interaction of a PR2 robot with randomly organized tabletop scenes. The model is designed to propose meaningful push actions based on over-segmented RGB-D images. We… 
[PDF] Semantic Reader
63 Citations
Highly Influential Citations
4
Background Citations
35
Methods Citations
15

63 Citations

Learning Goal-Oriented Non-Prehensile Pushing in Cluttered Scenes

This paper applies deep reinforcement learning to generate pushing actions for a robotic manipulator acting on a planar surface where objects have to be pushed to goal locations while avoiding other items in the same workspace.

Split Deep Q-Learning for Robust Object Singulation*

This paper proposes a pushing policy aiming at singulating the target object from its surrounding clutter, by means of lateral pushing movements of both the neighboring objects and thetarget object until sufficient ’grasping room’ has been achieved.

Total Singulation With Modular Reinforcement Learning

A modular reinforcement learning method which uses continuous actions to totally singulate the target object from its surrounding clutter, and which considerably outperforms the state-of-the-art methods in the singulation task.

A few-shot learning framework for planar pushing of unknown objects

A new large planar pushing dataset that contains a wide range of simulated objects and a novel representation for pushing primitives for the data-driven prediction model and a computation efficient planning method that employs a heuristic to reduce the possibility of making sliding contact between the pusher and the object.

Self-Supervised Interactive Object Segmentation Through a Singulation-and-Grasping Approach

A robot learning approach to actively interact with novel objects and collect each object’s training label for further tuning to improve the segmentation model performance, while avoiding the time-consuming process of manually labeling a dataset.

Graph-based Cluttered Scene Generation and Interactive Exploration using Deep Reinforcement Learning

A novel method to teach a robotic agent to interactively explore cluttered yet structured scenes, such as kitchen pantries and grocery shelves, by leveraging the physical plausibility of the scene by incorporating a novel scene grammar to represent structured clutter is introduced.

Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter

A novel Deep RL procedure is presented that combines i) teacher-aided exploration, ii) a critic with privileged information, and iii) mid-level representations, resulting in sample efficient and effective learning for the problem of uncovering a target object occluded by a heap of unknown objects.

Push-to-See: Learning Non-Prehensile Manipulation to Enhance Instance Segmentation via Deep Q-Learning

This paper introduces the first interactive segmentation model in the literature that can autonomously enhance the instance segmentation of such challenging scenes as a whole via optimising a Q-value function that predicts appropriate pushing actions for singulation.

Deep Reinforcement Learning-Based Robotic Grasping in Clutter and Occlusion

A multi-view change observation-based approach (MV-COBA) to overcome the challenges associated with robotic grasping in both clutter and occlusion scenarios and achieves an average grasp success rate of 83.6%, 86.3%, and 97.8% in the cluttered, well-ordered object, and occLusion scenarios, respectively.

Object and Relation Centric Representations for Push Effect Prediction

This paper proposes a graph neural network based framework for effect prediction and parameter estimation of pushing actions by modeling object relations based on contacts or articulations and demonstrates 6D effect prediction in the lever-up action in the context of robot-based hard-disk disassembly.
...

References

SHOWING 1-10 OF 34 REFERENCES

Perceiving, learning, and exploiting object affordances for autonomous pile manipulation

The results show that the learning-based approach for pile manipulation outperforms a common sense heuristic as well as a random strategy, and is on par with human action selection.

Learning to Manipulate Unknown Objects in Clutter by Reinforcement

A fully autonomous robotic system for grasping objects in dense clutter is presented, which learns online, from scratch, to manipulate the objects by trial and error using stochastic transitions between the observed states, using nonparametric density estimation.

Probabilistic Segmentation and Targeted Exploration of Objects in Cluttered Environments

Evaluations show that the proposed information-theoretic approach allows a robot to efficiently determine the composite structure of its environment, and the probabilistic model allows straightforward integration of multiple modalities, such as movement data and static scene features.

Deep visual foresight for planning robot motion

  • Chelsea FinnS. Levine
  • Computer Science
    2017 IEEE International Conference on Robotics and Automation (ICRA)
  • 2017
This work develops a method for combining deep action-conditioned video prediction models with model-predictive control that uses entirely unlabeled training data and enables a real robot to perform nonprehensile manipulation — pushing objects — and can handle novel objects not seen during training.

Guided pushing for object singulation

This work proposes a novel method for a robot to separate and segment objects in a cluttered tabletop environment by using the robot arm to perform pushing actions specifically selected to test whether particular visible edges correspond to object boundaries.

Metric learning for generalizing spatial relations to new objects

This paper addresses the problem of learning spatial relations by introducing a novel method from the perspective of distance metric learning that enables a robot to reason about the similarity between pairwise spatial relations, thereby enabling it to use its previous knowledge when presented with a new relation to imitate.

The Curious Robot: Learning Visual Representations via Physical Interactions

This work builds one of the first systems on a Baxter platform that pushes, pokes, grasps and observes objects in a tabletop environment, with each datapoint providing supervision to a shared ConvNet architecture allowing us to learn visual representations.

Segmentation and learning of unknown objects through physical interaction

It is shown that the learned model, in combination with the proposed segmentation process, allows robust object recognition in cluttered scenes.

A Framework for Push-Grasping in Clutter

This work introduces a framework for planning in clutter that uses a library of actions inspired by human strategies that succeeds where traditional grasp planners fail, and works under high uncertainty by utilizing the funneling effect of pushing.

Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

  • Lerrel PintoA. Gupta
  • Computer Science
    2016 IEEE International Conference on Robotics and Automation (ICRA)
  • 2016
This paper takes the leap of increasing the available training data to 40 times more than prior work, leading to a dataset size of 50K data points collected over 700 hours of robot grasping attempts, which allows us to train a Convolutional Neural Network for the task of predicting grasp locations without severe overfitting.