Sensory substitution and the human–machine interface

  title={Sensory substitution and the human–machine interface},
  author={Paul Bach-y-Rita and Stephen W. Kercel},
  journal={Trends in Cognitive Sciences},

Figures from this paper

Substituting the Senses

This chapter considers what philosophers might learn about the nature of the senses from the neuroscience of sensory substitution, and shows how sensory substitution devices work by exploiting the cross-modal plasticity of sensory cortex.

Cognitive and brain mechanisms in sensory substitution of vision: a contribution to the study of human perception.

This paper addresses some of the major questions raised by sensory substitution, demonstrates how the study of sensory substitution enhances the understanding of human perception and brain plasticity and provides an overview of rehabilitation potentialities.

Multisensory and spatial processes in sensory substitution.

  • M. Auvray
  • Psychology, Biology
    Restorative neurology and neuroscience
  • 2019
The factors underlying the learning of sensory substitution devices, their adequacy to the target population, and ways of improving their design by the use of crossmodal correspondences and by taking into account individual differences in the used reference frames are focused on.

Beyond sensory substitution--learning the sixth sense.

The results indicate that the sensory information provided by the belt is processed and boosts performance, and if inconsistent with other sensory signals leads to variable performance, does interact with the vestibular nystagmus and in half of the experimental subjects leads to qualitative changes of sensory experience.

Sensory Substitution and Perceptual Learning

  • K. Connolly
  • Biology
    Sensory Substitution and Augmentation
  • 2018
This chapter explores ways in which the extensive literature on perceptual learning can be applied to help improve sensory substitution devices and proposes a novel empirical test to help resolve a philosophical debate.

Sensory Substitution Systems

This paper presents several current applications in the fields of accessibility and sensory augmentation and a general review of the history and the physiological and technological principles of sensory substitution systems is given.

Other ways of seeing: From behavior to neural mechanisms in the online “visual” control of action with sensory substitution

It appears that the use of sensory substitution devices utilizes the neural visual system; this suggests the hypothesis that sensory substitution draws on the same underlying mechanisms as unimpaired visual control of action.

Sensory Substitution and Augmentation - What's Happening "Under the Hood" in Our Brain

This work will use the example of visual-to-auditory sensory substitution devices for the blind to discuss what happens "under the hood", how the use of sensory substitution actually appears to work and what this has revealed about the authors' brain organization in general, leading to theories such as the view of the brain as a task machine.

Sensory augmentation: integration of an auditory compass signal into human perception of space

It is argued that it is the reliability of the magnetic signal that allows vestibular spatial recalibration, and the coding scheme mimicking sensorimotor contingencies of distal sounds that permits fast integration, and it is proposed that contingency-mimetic feedback has great potential for creating sensory augmentation devices that achieve fast and genuinely perceptual experiences.

Neural Correlates of Sensory Substitution in Vestibular Pathways following Complete Vestibular Loss

Together the results provide direct evidence that the substitution by extravestibular inputs in vestibular pathways provides a neural correlate for the improvements in gaze stability that are observed following the total loss of vestibul inputs.



Sensori-"motor" coupling by observed and imagined movement

Sensory systems are associated with motor systems for perception. In the absence of motor control over the orientation of the sensory input, a person may have no idea from where the information is

Closing an open-loop control system: vestibular substitution through the tongue.

It is demonstrated that head-body postural coordination can be restored by means of vestibular substitution using a head-mounted accelerometer and a brain-machine interface that employs a unique pattern of electrotactile stimulation on the tongue.

Brain plasticity: ‘visual’ acuity of blind persons via the tongue

A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution

An experimental prototype based on a personal computer connected to a miniature head-fixed video camera and to headphones which has so far demonstrated prototype usefulness for pattern recognition and an integrated circuit of this system is to be developed.

Vision Substitution by Tactile Image Projection

WE describe here a vision substitution system which is being developed as a practical aid for the blind and as a means of studying the processing of afferent information in the central nervous

Auditory Coding of Visual Patterns for the Blind

Recognition tasks of simple visual patterns have been used to assess an early visual – auditory sensory-substitution system, consisting of the coupling of a rough model of the human retina with an

Occipital Activation by Pattern Recognition in the Early Blind Using Auditory Substitution for Vision

The results suggest that activity of the extrastriate visual cortex of early blind subjects can be modulated and bring additional evidence that early visual deprivation leads to cross-modal cerebral reorganization.

A sensorimotor account of vision and visual consciousness.

It is proposed that seeing is a way of acting, which provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities.