A Soft Humanoid Hand with In-Finger Visual Perception

  title={A Soft Humanoid Hand with In-Finger Visual Perception},
  author={Felix Hundhausen and Julia Starke and Tamim Asfour},
  journal={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
We present a novel underactued humanoid five finger soft hand, the KIT Finger-Vision Soft Hand, which is equipped with cameras in the fingertips and integrates a high performance embedded system for visual processing and control. We describe the actuation mechanism of the hand and the tendon-driven soft finger design with internally routed high-bandwidth flat-flex cables. For efficient on-board parallel processing of visual data from the cameras in each fingertip, we present a hybrid embedded… 

Figures and Tables from this paper

Fast Reactive Grasping with In-Finger Vision and In-Hand FPGA-accelerated CNNs
An FPGA-based, in-hand integrated, embedded system for processing visual data captured by the five in-finger cameras while avoiding high bandwidth raw data streaming via the robots real-time data bus.
Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis
This work presents an eye-inhand learning-based approach for hand pre-shape classification from RGB sequences, and develops a sensorized setup to acquire real human grasping sequences for benchmarking and shows that models trained with the synthetic dataset achieve better generalization performance than models trained on real data.
Detecting Grasp Phases and Adaption of Object-Hand Interaction Forces of a Soft Humanoid Hand Based on Tactile Feedback
A new soft humanoid hand equipped with a multimodal sensor system in each finger and a human-inspired grasp-phases controller that is able to detect the different phases of a grasping and manipulation task, adapt interaction forces with the manipulated object and balance the force distribution in both precision and power grasps based on tactile feedback is presented.
Designing Prosthetic Hands With Embodied Intelligence: The KIT Prosthetic Hands
The developed prostheses provide intelligent mechatronics including adaptive actuation, multi-modal sensing and on-board computing resources to enable autonomous and intuitive control and exemplarily show the suitability of the multi- modal sensor system for sensory-based, semi-autonomous grasping in daily life activities.
RBO Hand 3 - A Platform for Soft Dexterous Manipulation
The RBO Hand 3 is designed to enable dexterous manipulation, to facilitate transfer of insights about human dexterity, and to serve as a robust research platform for extensive realworld experiments by combining many degrees of actuation with intrinsic compliance, replicating relevant functioning of the human hand, and by combining robust components in a modular design.
Binary-LoRAX: Low-Latency Runtime Adaptable XNOR Classifier for Semi-Autonomous Grasping with Prosthetic Hands
Binary-LoRAX is proposed, a low-latency runtime adaptable classifier for the semi-autonomous grasping task of prosthetic hands which offload the classification task to an efficient binary neural network accelerator which performs high-throughput XNOR operations on digital signal processing (DSP) blocks.


Design and Experimental Research of Pneumatic Soft Humanoid Robot Hand
A pneumatic soft humanoid robot hand which imitates the shapes and gestures of human hand is introduced in this article and can perform complex gestures and grasp objects with different shapes.
Vision-Based Tactile Perception for Soft Robotic Hand*
Results show the effectiveness of the proposed method and the superior performance of vision-based tactile perception.
Development of the NAIST-Hand with Vision-based Tactile Fingertip Sensor
A multifingered robotic hand eNAIST-Handf and a grip force control by slip margin feedback and a new tactile sensor designed to active the proposed control method by the NAIST- hand are introduced.
The KIT Prosthetic Hand: Design and Control
The KIT prosthetic hand is presented, a novel five-finger 3D printed hand prosthesis, with its underactuated mechanism, sensors and embedded control system, based on the TUAT/Karlsruhe mechanism with two motors actuating 10 degrees of freedom.
Learning a visuomotor controller for real world robotic grasping using simulated depth images
This paper proposes an approach to learning a closed-loop controller for robotic grasping that dynamically guides the gripper to the object and finds that this approach significantly outperforms the baseline in the presence of kinematic noise, perceptual errors and disturbances of the object during grasping.
Resource-Aware Object Classification and Segmentation for Semi-Autonomous Grasping with Prosthetic Hands
A visual perception system to extract scene information for semi-autonomous hand-control that allows minimizing required command complexity and leads to more intuitive and effortless control and an evolutionary algorithm to optimize CNN architecture regarding accuracy and hardware resource demand is presented.
A compliant, underactuated hand for robust manipulation
The iRobot-Harvard-Yale (iHY) Hand is introduced, an underactuated hand driven by five actuators that is capable of performing a wide range of grasping and in-hand repositioning tasks.
Robotic grasp control with high-resolution combined tactile and proximity sensing
The robotic motion for searching, approaching and grasping can be controlled based on sensor information obtained from only the proposed sensing device, and the combined sensing approach enables an adaptive grasping motion of the robotic hand.
Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
This work decomposes the end-to-end system into a vision module and a closed-loop controller module that is robust with respect to discrepancies between the dynamic model of the simulated and real robot, and achieves a 90% success rate in grasping a tiny sphere with a real robot.
A novel type of compliant and underactuated robotic hand for dexterous grasping
RBO Hand 2 is presented, a highly compliant, underactuated, robust, and dexterous anthropomorphic hand that is inexpensive to manufacture and the morphology can easily be adapted to specific applications, and it is demonstrated that complex grasping behavior can be achieved with relatively simple control.