A Taxonomy of Sounds in Virtual Reality

  title={A Taxonomy of Sounds in Virtual Reality},
  author={Dhruv Jain and Sasa Junuzovic and Eyal Ofek and Mike Sinclair and John Porter and Chris Yoon and Swetha Machanavajhala and Meredith Ringel Morris},
  journal={Designing Interactive Systems Conference 2021},
Virtual reality (VR) leverages human sight, hearing and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, information conveyed through sound may not be accessible. To help with future design of accessible VR sound representations for DHH users, this paper contributes a consistent language and structure for representing sounds in VR. Using two studies, we report on the design and evaluation of a novel taxonomy for VR sounds. Study 1 included interviews with… 

Tables from this paper

Towards Sound Accessibility in Virtual Reality
This paper provides a first comprehensive investigation of sound accessibility in VR, including a design space for developing visual and haptic substitutes of VR sounds to support DHH users and prototypes illustrating several points within the design space.
Exploring Manipulating In-VR Audio To Facilitate Verbal Interactions Between VR Users And Bystanders
audio manipulations can be used to significantly improve a VR user’s auditory awareness at the cost of reducing sense of presence in VR, however, they also identify a subset of participants who prioritised increasing awareness no matter the cost to presence.


Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds
A novel white cane controller is examined that enables navigation without vision of large virtual environments with complex architecture, such as winding paths and occluding walls and doors and employs a lightweight three-axis brake mechanism to provide large-scale shape of virtual objects.
Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation
Canetroller, a haptic cane controller that simulates white cane interactions, enabling people with visual impairments to navigate a virtual environment by transferring their cane skills into the virtual world, was showed to be a promising tool that enabled visually impaired participants to navigate different virtual spaces.
Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing
This paper design and evaluate visualizations for spatially locating sound on a head-mounted display (HMD) and developed eight high-level visual sound feedback dimensions, reaffirm past work on challenges faced by persons with hearing loss in group conversations, provide support for the general idea of sound awareness visualizations on HMDs, and reveal preferences for specific design options.
Deaf and Hard-of-hearing Individuals' Preferences for Wearable and Mobile Sound Awareness Technologies
Findings related to sound type, full captions vs. keywords, sound filtering, notification styles, and social context provide direct guidance for the design of future mobile and wearable sound awareness systems.
Exploring Virtual Reality to Enable Deaf or Hard of Hearing Accessibility in Live Theaters: A Case Study
Evaluated the efficiency of a language processing system, which makes use of VR technology combined with AI implementations for automatic speech recognition (ASR), sentence prediction and spelling correction, which demonstrates good overall results regarding DHH understanding and satisfaction along entire play sessions.
Evaluating non-speech sound visualizations for the deaf
An iterative investigation of peripheral, visual displays of ambient sounds, providing valuable information about the sound awareness needs of the deaf and can help to inform further design of such applications.
Virtual Showdown: An Accessible Virtual Reality Game with Scaffolds for Youth with Visual Impairments
A VR game based on an accessible real-world game called Showdown, where people use their hearing to locate and hit a ball against an opponent and developed Verbal and Verbal/Vibration Scaffolds to teach people how to play Virtual Showdown.
A Critical Examination of Virtual Reality Technology in the Context of the Minority Body
Findings show that VR is an inherently ableist technology that assumes a ‘corporeal standard’ (i.e., an ‘ideal’, non-disabled human body), and fails to adequately accommodate disabled people.
Virtual Reality as a Tool for Improving Spatial Rotation among Deaf and Hard-of-Hearing Children
Results clearly indicate that practicing with VR 3D spatial rotations significantly improved the children's performance of spatial rotation, which enhanced their ability to perform better in other intellectual skills as well as in their sign language skills.
Exploring Sound Awareness in the Home for People who are Deaf or Hard of Hearing
A general interest in smarthome-based sound awareness systems is suggested particularly for displaying contextually aware, personalized and glanceable visualizations but key concerns arose related to privacy, activity tracking, cognitive overload, and trust.