Towards vision-based robotic skins: a data-driven, multi-camera tactile sensor

  title={Towards vision-based robotic skins: a data-driven, multi-camera tactile sensor},
  author={Camill Trueeb and Carmelo Sferrazza and Raffaello D’Andrea},
  journal={2020 3rd IEEE International Conference on Soft Robotics (RoboSoft)},
This paper describes the design of a multi-camera optical tactile sensor that provides information about the contact force distribution applied to its soft surface. This information is contained in the motion of spherical particles spread within the surface, which deforms when subject to force. The small embedded cameras capture images of the different particle patterns that are then mapped to the three-dimensional contact force distribution through a machine learning architecture. The design… Expand
Single to Multi: Data-Driven High Resolution Calibration Method for Piezoresistive Sensor Array
Experimental results show that the proposed novel supervised learning framework, Local Message Passing Network, can expand prior knowledge of single touch data to calibrate multi-touch sensor inputs into high resolution pressure maps. Expand
Development of Soft Tactile Sensor for Force Measurement and Position Detection
A novelly designed soft sensor that can simultaneously estimate the contact force and contact location and is Inspired by humans’ skin, which contains multi-layers of receptors. Expand
F-TOUCH Sensor: Concurrent Geometry Perception and Multi-Axis Force Measurement
Force and tactile sensing has experienced a surge of interest over recent decades. It conveys a range of information through physical interaction, including force, pressure, texture and vibration.Expand
Viko: An Adaptive Gecko Gripper with Vision-based Tactile Sensor
Viko is presented, an adaptive gecko gripper that utilizes visionbased tactile sensors to monitor contact state and provides high-resolution real-time measurements of contact area and shear force. Expand
3D Contact Position Estimation of Image-Based Areal Soft Tactile Sensor with Printed Array Markers and Image Sensors
The purpose of this study is to estimate 3D contact position of a novel image-based areal soft tactile sensor (IASTS) using printed array markers and multiple cameras and to ensure the stability of the estimated contact positions a Kalman filter was developed. Expand
Sim-to-real for high-resolution optical tactile sensing: From images to 3D contact force distributions
A strategy to generate tactile images in simulation for a vision-based tactile sensor based on an internal camera that tracks the motion of spherical particles within a soft material using an artificial neural network entirely trained on synthetic data avoiding the need for real-world data collection. Expand
The Terminal Trajectory Drive and Control of a Three-Channel Soft Actuator
This paper aims to do a coupling control of three paralleled channels’ pressures and establish a pneumatic drive model of extruding and bending, and then present the terminal trajectory control system. Expand


Ground Truth Force Distribution for Learning-Based Tactile Sensing: A Finite Element Approach
The method presented in this article uses a finite element model to obtain ground truth data for the three-dimensional force distribution and is applied to a vision-based tactile sensor, which aims to reconstruct the contact force distribution purely from images. Expand
A Novel Dynamic-Vision-Based Approach for Tactile Sensing Applications
This approach is the first event-based tactile sensor which utilizes the recent technology of neuromorphic cameras and provides higher sensitivity, a lower latency, and less computational and power consumption compared to other conventional vision-based techniques. Expand
Design, Motivation and Evaluation of a Full-Resolution Optical Tactile Sensor
The vision-based tactile sensor proposed in this article exploits the extremely high resolution of modern image sensors to reconstruct the normal force distribution applied to a soft material, whose deformation is observed on the camera images. Expand
Development of a Vision-Based Soft Tactile Muscularis
The developed tactile muscularis can be integrated on the standard articulate robotic arm for enhancement of the robot's interaction with the surroundings under physical acting and pave a way for the development of a friendly robot with proper tactile sensing for safe and comfortable interaction with humans. Expand
Exoskeleton-covered soft finger with vision-based proprioception and exteroception
This work develops a novel exoskeleton-covered soft finger with embedded cameras and deep learning methods that enable high-resolution proprioceptive sensing and rich tactile sensing and assembles 2 of the fingers together to form a robotic gripper and successfully perform a bar stock classification task, which requires both shape and tactile information. Expand
Online appendix - On the thickness of vision-based tactile sensors
  • 2019. [Online]. Available:
  • 2019
Recent progress in tactile sensing and sensors for robotic manipulation: can we turn tactile sensing into vision?1
This paper surveys recently published literature on tactile sensing in robotic manipulation to understand effective strategies for using tactile sensing and the issues involved in tactile sensing. ItExpand
Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation
A lightweight, easy-to-build, highly compliant dense geometry sensor and end effector that comprises an inflated latex membrane with a depth sensor behind it and its capabilities through example tasks including tactile-object classification, pose estimation and tracking, and nonprehensile object manipulation is presented. Expand
Tactile Image Sensors Employing Camera: A Review
An overview of tactile image sensors employing a camera is provided with a focus on the sensing principle, typical design, and variation in the sensor configuration. Expand
Transfer learning for vision-based tactile sensing
In order to reduce the training times and the need for large datasets, a calibration procedure is proposed to transfer the acquired knowledge across multiple sensors while maintaining satisfactory performance. Expand