Connecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials

  title={Connecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials},
  author={Wenzhen Yuan and Shaoxiong Wang and Siyuan Dong and E. Adelson},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
For machines to interact with the physical world, they must understand the physical properties of objects and materials they encounter. We use fabrics as an example of a deformable material with a rich set of mechanical properties. A thin flexible fabric, when draped, tends to look different from a heavy stiff fabric. It also feels different when touched. Using a collection of 118 fabric samples, we captured color and depth images of draped fabrics along with tactile data from a high-resolution… Expand
Teaching Cameras to Feel: Estimating Tactile Physical Properties of Surfaces From Images
Active Clothing Material Perception Using Tactile Sensing and Deep Learning
Deep Visuo-Tactile Learning: Estimation of Tactile Properties from Images
Deep Visuo-Tactile Learning: Estimation of Material Properties from Images
ViTac: Feature Sharing Between Vision and Tactile Sensing for Cloth Texture Recognition
Connecting Touch and Vision via Cross-Modal Prediction
Learning Intuitive Physics with Multimodal Generative Models
Learning to Identify Object Instances by Touch: Tactile Recognition via Multimodal Matching


Can you see what you feel? Color and folding properties affect visual-tactile material discrimination of fabrics.
Estimating the Material Properties of Fabric from Video
Joint embeddings of shapes and images via CNN image purification
Retrographic sensing for the measurement of surface texture and shape
Visual perception of materials and their properties
Sensing and Recognizing Surface Textures Using a GelSight Sensor
  • R. Li, E. Adelson
  • Computer Science
  • 2013 IEEE Conference on Computer Vision and Pattern Recognition
  • 2013
A Two-Level Generative Model for Cloth Representation and Shape from Shading
  • Feng Han, Song-Chun Zhu
  • Mathematics, Computer Science
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2007
Estimating object hardness with a GelSight touch sensor
On seeing stuff: the perception of materials by humans and machines
  • E. Adelson
  • Computer Science, Engineering
  • IS&T/SPIE Electronic Imaging
  • 2001
Capturing and animating occluded cloth