From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor

@article{Lepora2019FromPT,
  title={From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor},
  author={Nathan F. Lepora and Alex Church and Conrad de Kerckhove and Raia Hadsell and John Lloyd},
  journal={IEEE Robotics and Automation Letters},
  year={2019},
  volume={4},
  pages={2101-2107}
}
Deep learning has the potential to have same the impact on robot touch as it has had on robot vision. Optical tactile sensors act as a bridge between the subjects by allowing techniques from vision to be applied to touch. In this letter, we apply deep learning to an optical biomimetic tactile sensor, the TacTip, which images an array of papillae (pins) inside its sensing surface analogous to structures within human skin. Our main result is that the application of a deep convolutional neural… Expand
Sim-to-Real Transfer for Optical Tactile Sensing
TLDR
A model for soft body simulation which was implemented using the Unity physics engine, and a neural network was trained to predict the locations and angles of edges when in contact with the sensor to accurately predict edges with less than 1 mm prediction error in real-world testing. Expand
Optimal Deep Learning for Robot Touch
This article illustrates the application of deep learning to robot touch by considering a basic yet fundamental capability: estimating the relative pose of part of an object in contact with a tactileExpand
Optimal Deep Learning for Robot Touch: Training Accurate Pose Models of 3D Surfaces and Edges
This article illustrates the application of deep learning to robot touch by considering a basic yet fundamental capability: estimating the relative pose of part of an object in contact with a tactileExpand
Convolutional Autoencoder for Feature Extraction in Tactile Sensing
TLDR
This letter presents a method of dimensionality reduction of an optical-based tactile sensor image output using a convolutional neural network encoder structure that allows simultaneous online deployment of multiple simple perception algorithms on a common set of black-box features. Expand
Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear
TLDR
A supervised convolutional deep neural network model is proposed that learns to disentangle the components of sensor deformations caused by contact geometry from those due to sliding-induced shear. Expand
Pose-Based Servo Control with Soft Tactile Sensing
TLDR
A new way of controlling robots using soft tactile sensors: pose-based tactile servo (PBTS) control is described, to embed a tactile perception model for estimating the sensor pose within a servo control loop that is applied to local object features such as edges and surfaces. Expand
A Simple Robotic Fingertip Sensor Using Imaging and Shallow Neural Networks
TLDR
A low-cost fingertip sensor that can simultaneously measure the contact position and normal, the 3-D contact force, and the local radius of curvature of the object in contact with the fingertip is presented. Expand
Soft Biomimetic Optical Tactile Sensing With the TacTip: A Review
  • N. Lepora
  • Computer Science
  • IEEE Sensors Journal
  • 2021
TLDR
This article reviews the BRL TacTip as a prototypical example of a SoftBOT (Soft Biomimetic Optical Tactile) sensor and discusses the relation between artificial skin morphology and the transduction principles of human touch. Expand
Uncertainty-aware deep learning for robot touch: Application to Bayesian tactile servo control
TLDR
It is demonstrated that uncertainty-aware DL can improve the pose estimation over deterministic DL methods and improve the tactile servo control. Expand
Bionic Compound Eye-Inspired High Spatial and Sensitive Tactile Sensor
TLDR
A novel high-density arrayed and curved 3-D tactile sensor based on bionic compound eye with the density up to 42 units/cm is proposed in this article and will provide a technical basis for force measurement of surgical robots and manipulators. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 49 REFERENCES
Tactile object recognition using deep learning and dropout
TLDR
This paper aims for multimodal object recognition by power grasping of objects with an unknown orientation and position relation to the hand by using a denoising autoencoder with dropout compared to traditional neural networks. Expand
Exploiting Sensor Symmetry for Generalized Tactile Perception in Biomimetic Touch
TLDR
The generalization method is applied to the TacTip v2, a three-dimensional printed optical tactile sensor with internal pins acting as taxels arranged with a 12-fold rotational symmetry, and is able to generalize tactile stimuli to new orientations. Expand
Shape-independent hardness estimation using deep learning and a GelSight tactile sensor
TLDR
This work introduces a novel method for hardness estimation, based on the GelSight tactile sensor, and it is shown that the neural net model can estimate the hardness of objects with different shapes and hardness ranging from 8 to 87 in Shore 00 scale. Expand
More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch
TLDR
An end-to-end action-conditional model that learns regrasping policies from raw visuo-tactile data and outperforms a variety of baselines at estimating grasp adjustment outcomes, selecting efficient grasp adjustments for quick grasping, and reducing the amount of force applied at the fingers, while maintaining competitive performance. Expand
Robust material classification with a tactile skin using deep learning
  • S. S. Baishya, B. Bäuml
  • Engineering, Computer Science
  • 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2016
TLDR
This paper shows that material classification purely based on the spatio-temporal signal of a flexible tactile skin can be robustly performed in a real world setting, and compares different classification algorithms and feature sets that arebased on the signal's Fourier spectrum. Expand
Development of a tactile sensor based on biologically inspired edge encoding
TLDR
Initial results presented here show the design to be a very capable, highly sensitive sensor as well as a very practical, affordable and scalable robotic fingertip. Expand
Exploratory Tactile Servoing With Active Touch
TLDR
This work focuses on a prototypical task of tactile exploration over surface features such as edges or ridges, which is a principal exploratory procedure of humans to recognize object shape, and brings together active perception and haptic exploration as instantiations of a common active touch algorithm. Expand
Superresolution with an optical tactile sensor
TLDR
A novel optical sensor design (the TacTip) capable of achieving 40-fold localization superresolution to 0.1mm accuracy is demonstrated, comparable to the best perceptual hyperacuity in humans. Expand
Biomimetic Active Touch with Fingertips and Whiskers
  • N. Lepora
  • Engineering, Computer Science
  • IEEE Transactions on Haptics
  • 2016
TLDR
Biomimetic active touch offers a common approach for biomimetic tactile sensors to accurately and robustly characterize and explore non-trivial, uncertain environments analogous to how animals perceive the natural world. Expand
Active contour following to explore object shape with robot touch
TLDR
This work proposes a control architecture that implements a perception-action cycle for the exploratory procedure, which allows the fingertip to react to tactile contact whilst regulating the applied contact force. Expand
...
1
2
3
4
5
...