Learning Bimanual Scooping Policies for Food Acquisition

  title={Learning Bimanual Scooping Policies for Food Acquisition},
  author={Jennifer Grannen and Yilin Wu and Suneel Belkhale and Dorsa Sadigh},
  booktitle={Conference on Robot Learning},
A robotic feeding system must be able to acquire a variety of foods. Prior bite acquisition works consider single-arm spoon scooping or fork skewering, which do not generalize to foods with complex geometries and deformabilities. For example, when acquiring a group of peas, skewering could smoosh the peas while scooping without a barrier could result in chasing the peas on the plate. In order to acquire foods with such diverse properties, we propose stabilizing food items during scooping using… 

Figures and Tables from this paper

In-Mouth Robotic Bite Transfer with Visual and Haptic Sensing

An essential capability of this platform is demonstrated: safe, comfortable, and effective transfer of a bite-sized food item from a utensil directly to the inside of a person's mouth.

Learning Tool Morphology for Contact-Rich Manipulation Tasks with Differentiable Simulation

This work presents an end-to-end framework to automatically learn tool morphology for contact-rich manipulation tasks by leveraging differentiable physics simulators and demonstrates the effectiveness of the method for designing new tools in several scenarios.

AutoBag: Learning to Open Plastic Bags and Insert Objects

A self-supervised learning framework where a dual-arm robot learns to recognize the handles and rim of plastic bags using UV-fluorescent markings and the AutoBag algorithm, where the robot uses the learned perception model to open a plastic bag through iterative manipulation.



Robot-Assisted Feeding: Generalizing Skewering Strategies Across Food Items on a Plate

A bite acquisition framework that takes the image of a full plate as an input, segments it into food items, and then applies the Skewering-Position-Action network (SPANet) to choose a target food item and a corresponding action so that the bite acquisition success rate is maximized.

Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding

This work proposes a modified linear contextual bandit framework augmented with post hoc context observed after action selection to empirically increase learning speed and reduce cumulative regret in autonomous robot-assisted feeding.

Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulation

This paper leverages human demonstrations to develop an autonomous chopsticks-equipped robotic manipulator and proposes two approaches to reduce covariate shift, neither of which requires access to an interactive expert or a model, unlike previous approaches.

Leveraging Multimodal Haptic Sensory Data for Robust Cutting

Using vibrations and force-torque feedback from the interactions to adapt the slicing motions and monitor for contact events and the robot is able to reliably cut through more than 20 different types of food items and even detect whether certain food items are fresh or old.

Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning

This work designs a “Connect Task”, where the aim is for two robot arms to pick up and attach two blocks with magnetic connection points, and finds that the system has 100% success rate at picking up blocks, and 65% success rates at the “ connect task”.

Towards Robotic Feeding: Role of Haptics in Fork-Based Food Manipulation

A set of classifiers for compliance-based food categorization from haptic and motion signals is proposed and compared with fixed position-control policies via a robot to highlight the importance of adapting the policy to the compliance of a food item.

Telemanipulation with Chopsticks: Analyzing Human Factors in User Demonstrations

Although participants rated teleoperation as the least comfortable and most difficult-to-use method, teleoperation enabled users to achieve the highest success rates on three out of five objects considered, and it is noticed that subjects quickly learned and adapted to the teleoperation interface.

Transfer Depends on Acquisition: Analyzing Manipulation Strategies for Robotic Feeding

The results show that an intelligent food item dependent skewering strategy improves the bite acquisition success rate and that the choice of skewering location and the fork orientation affects the ease of bite transfer sianificantly.

FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding

A self-supervised learning framework, FlingBot, that learns how to unfold a piece of fabric from arbitrary initial configurations using a pick, stretch, and fling primitive for a dualarm setup from visual observations is proposed and demonstrates the effectiveness of dynamic flinging actions for cloth unfolding.

Meal support system with spoon using laser range finder and manipulator

An autonomous meal support robot system that can handle non-rigid solid food with a laser range finder and a manipulator holding a spoon that can scoop foods with high success rate is presented.