A Personalizable Mobile Sound Detector App Design for Deaf and Hard-of-Hearing Users

  title={A Personalizable Mobile Sound Detector App Design for Deaf and Hard-of-Hearing Users},
  author={Danielle Bragg and Nicholas Huynh and Richard E. Ladner},
  journal={Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility},
Sounds provide informative signals about the world around us. In situations where non-auditory cues are inaccessible, it can be useful for deaf and hard-of-hearing people to be notified about sounds. Through a survey, we explored which sounds are of interest to deaf and hard-of-hearing people, and which means of notification are appropriate. Motivated by these findings, we designed a mobile phone app that alerts deaf and hard-of-hearing people to sounds they care about. The app uses training… 

Figures and Tables from this paper

Mobile Sound Recognition for the Deaf and Hard of Hearing
An exploratory study in the domain of assistive computing, eliciting requirements and presenting solutions to problems found in the development of an environmental sound recognition system, which aims to assist deaf and hard of hearing people in the perception of sounds.
Deaf and Hard-of-hearing Individuals' Preferences for Wearable and Mobile Sound Awareness Technologies
Findings related to sound type, full captions vs. keywords, sound filtering, notification styles, and social context provide direct guidance for the design of future mobile and wearable sound awareness systems.
Interest and Requirements for Sound-Awareness Technologies Among Deaf and Hard-of-Hearing Users of Assistive Listening Devices
It is found that users of assistive listening devices still often miss sounds and rely on other people to obtain information about them, and the importance of awareness of different types of sounds varied according to the environment and the form factor of the sound-awareness technology.
Home Environmental Sound Alert System for Deaf and Hard-of-Hearing Users
A home environmental sound alert system for deaf and hard-of-hearing users is presented, which detects the sound generated in the home environment, converts the sound into text, and provides this text to the user.
Prototype Design of Alert Device for Hearing Impaired Users
The proposed proto-type will reduce the dependency of hearing impaired users on other family members around them within the work environment at home and will improve their daily life activities through Information and Communication Technology (ICT).
ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users
ProtoSound is introduced, an interactive system for customizing sound recognition models by recording a few examples, thereby enabling personalized and fine-grained categories and discussing open challenges in personalizable sound recognition, including the need for better recording interfaces and algorithmic improvements.
Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing
To better understand how DHH users can drive personalization of their own assistive sound recognition tools, a three-part study with 14 DHH participants highlights a positive subjective experience when recording and interpreting training data in situ, but uncovers several key pitfalls unique to D HH users.
Evaluating Smartwatch-based Sound Feedback for Deaf and Hard-of-hearing Users Across Contexts
The findings characterize uses for vibration in multimodal sound awareness, both for push notification and for immediately actionable sound information displayed through vibrational patterns (tactons) in smartwatch feedback techniques.
UbiEar: Bringing Location-independent Sound Awareness to the Hard-of-hearing People with Smartphones
UbiEar, a smartphone-based acoustic event sensing and notification system, is designed and shown that UbiEar can assist the young DHH students in awareness of important acoustic events in their daily life.
HomeSound: An Iterative Field Deployment of an In-Home Sound Awareness System for Deaf or Hard of Hearing Users
HomeSound, an in-home sound awareness system for Deaf and hard of hearing (DHH) users, consists of a microphone and display, and uses multiple devices installed in each home, similar to the Echo Show or Nest Hub.


Can you see what i hear?: the design and evaluation of a peripheral sound display for the deaf
Two visual displays for providing awareness of environmental audio to deaf individuals are developed that support both monitoring and notification of sounds, support discovery of new sounds, and do not require a priori knowledge of sounds to be detected.
Evaluating non-speech sound visualizations for the deaf
An iterative investigation of peripheral, visual displays of ambient sounds, providing valuable information about the sound awareness needs of the deaf and can help to inform further design of such applications.
Scribe4Me: Evaluating a Mobile Sound Transcription Tool for the Deaf
A 2-week field study of an exploratory prototype of a mobile sound transcription tool for the deaf and hard-of-hearing shows that the approach is feasible, highlights particular contexts in which it is useful, and provides information about what should be contained in transcriptions.
A multifunction domestic alert system for the deaf-blind
An electronic system to alert a deaf-blind man to the occurrence of a variety of household sounds-such as a doorbell, telephone or a smoke alarm-and to inform him about the source of any such sound and to be effective, reliable, relatively inexpensive and adaptable to other potential users' requirements.
An efficient robust sound classification algorithm for hearing aids.
This work attempts to distinguish between three listening environment categories: speech in traffic noise, speech in babble, and clean speech, regardless of the signal-to-noise ratio, using an efficient robust sound classification algorithm based on hidden Markov models.
Deaf people communicating via SMS, TTY, relay service, fax, and computers in Australia.
Results showed that Short Message Service, telephone typewriters, TTY, voice/TTY relay services, fax, and e-mail were used regularly and Deaf users are discerning of the purposes for which they use each method.
Vibration signaling in mobile devices for emergency alerting: a study with deaf evaluators.
The U.S. government's recommended vibration signal for the CMAS, an important feature for access to emergency alerts by deaf persons, is supported by the results of the study.
Accurate and privacy preserving cough sensing using a low-cost microphone
A novel algorithm for detecting coughs from the audio stream of a mobile phone that allows cough sounds to be reconstructed from the feature set, but prevents speech from being reconstructed intelligibly.
Events Detection for an Audio-Based Surveillance System
The automatic shot detection system presented is based on a novelty detection approach which offers a solution to detect abnormality (abnormal audio events) in continuous audio recordings of public places and takes advantage of potential similarity between the acoustic signatures of the different types of weapons by building a hierarchical classification system.
Recognition of acoustical alarm signals for the profoundly deaf using hidden Markov models
  • S. Oberle, A. Kaelin
  • Computer Science
    Proceedings of ISCAS'95 - International Symposium on Circuits and Systems
  • 1995
A new acoustical alarm signal recognition scheme for tactile hearing aids using hidden Markov models (HMM's) is presented. In particular, a maximum likelihood classifier is proposed where the