Clarity-2021 Challenges: Machine Learning Challenges for Advancing Hearing Aid Processing

  title={Clarity-2021 Challenges: Machine Learning Challenges for Advancing Hearing Aid Processing},
  author={Simone Graetzer and Jon Barker and Trevor J. Cox and Michael A. Akeroyd and John F. Culling and Graham Naylor and Eszter Porter and Rhoddy Viveros Mu{\~n}oz},
In recent years, rapid advances in speech technology have been made possible by machine learning challenges such as CHiME, REVERB, Blizzard, and Hurricane. In the Clarity project, the machine learning approach is applied to the problem of hearing aid processing of speech-in-noise, where current technology in enhancing the speech signal for the hearing aid wearer is often ineffective. The scenario is a (simulated) cuboid-shaped living room in which there is a single listener, a single target… 

Figures and Tables from this paper

Listening with Googlears: Low-Latency Neural Multiframe Beamforming and Equalization for Hearing Aids

Understanding speech in the presence of noise with hearing aids can be challenging. Here we describe our entry, submission E003, to the 2021 Clarity Enhancement Challenge Round 1 (CEC1), a machine

Speech Intelligibility Prediction for Hearing-Impaired Listeners with the LEAP Model

The proposed non-intrusive model outperforms the intrusive MBSTOI for both data sets and achieves a prediction accuracy of 25.88% and a correlation of 0.54.

Optimising Hearing Aid Fittings for Speech in Noise with a Differentiable Hearing Loss Model

A differentiable hearing loss model is proposed and used to optimise fittings with back-propagation and the objective evaluation shows the advantages of optimised custom fittings over general prescriptive fittings.

ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding

Results show that the integration of SE front-ends with back-end tasks is a promising research direction even for tasks besides ASR, especially in the multi-channel scenario.

MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids

Improving the user’s hearing ability to understand speech in noisy environments is critical to the development of hearing aid (HA) devices. For this, it is important to derive a metric that can

Exploiting Hidden Representations from a DNN-based Speech Recogniser for Speech Intelligibility Prediction in Hearing-impaired Listeners

This work leverages the hidden representations from DNN-based ASR as features for speech intelligibility prediction in hearing-impaired listeners and shows that the proposed method could make better prediction than a widely used short-time objective intelligibility (STOI) based binaural measure.

Auditory-Based Data Augmentation for end-to-end Automatic Speech Recognition

The results show that the proposed augmentation methods can bring statistically significant improvement on the performance of the state-of-the-art SpecAugment.

An End-to-End Deep Learning Speech Coding and Denoising Strategy for Cochlear Implants

A deep learning speech denoising sound coding strategy that estimates the CI electric stimulation patterns out of the raw audio data captured by the micro-phone, performing end-to-end CI processing is proposed.

A Neural-Network Framework for the Design of Individualised Hearing-Loss Compensation

A differentiable deep-neural-network (DNN) framework that can be used to train DNN-based HA models based on biophysical auditory-processing differences between normal-hearing and hearing-impaired models is proposed.

Deep Multi-Frame MVDR Filtering for Binaural Noise Reduction

To improve speech intelligibility and speech quality in noisy environments, binaural noise reduction algorithms for head-mounted assistive listening devices are of crucial importance. Several



A summary of the REVERB challenge: state-of-the-art and remaining challenges in reverberant speech processing research

The REVERB challenge is described, which is an evaluation campaign that was designed to evaluate such speech enhancement and ASR techniques to reveal the state-of-the-art techniques and obtain new insights regarding potential future research directions.

Intelligibility-enhancing speech modifications: the hurricane challenge

Surprisingly, for most conditions the largest gains were observed for noise-independent algorithms, suggesting that performance in this task can be further improved by exploiting information in the masking signal.

A binaural short time objective intelligibility measure for noisy and enhanced speech

A binaural extension of the STOI measure, which predicts bINAural advantage using a modified equalization cancellation (EC) stage, is proposed, which is able to predict the advantage of spatial separation between a speech target and a speech shaped noise (SSN) interferer.

CHiME-6 Challenge: Tackling Multispeaker Speech Recognition for Unsegmented Recordings

Of note, Track 2 is the first challenge activity in the community to tackle an unsegmented multispeaker speech recognition scenario with a complete set of reproducible open source baselines providing speech enhancement, speaker diarization, and speech recognition modules.

Use of a loudness model for hearing aid fitting: II. Hearing aids with multi-channel compression.

It was concluded that the initial fitting method gives reasonable starting values for the fine tuning of a multi-channel compression hearing aid and leads to satisfactory loudness impressions in everyday life and to high speech intelligibility over a wide range of levels.

An international comparison of long‐term average speech spectra

The long‐term average speech spectrum (LTASS) and some dynamic characteristics of speech were determined for 12 languages: English (several dialects), Swedish, Danish, German, French (Canadian),

Simulation of the effect of threshold elevation and loudness recruitment combined with reduced frequency selectivity on the intelligibility of speech in noise.

Evidence is consistent with previous evidence indicating that only part of the decrease of performance produced by actual cochlear hearing loss can be compensated by conventional linear hearing aids.

Open Master Hearing Aid (openMHA)—An integrated platform for hearing aid research

This contribution introducesopenMHA and discusses the usage and possible application scenarios of the portable openMHA setup in hearing research.

Open-source Multi-speaker Corpora of the English Accents in the British Isles

A dataset of transcribed high-quality audio of English sentences recorded by volunteers speaking with different accents of the British Isles is presented, intended for linguistic analysis as well as use for speech technologies.