Download PDF
CC BY-NC-ND 4.0 · Semin Hear 2021; 42(03): 260-281
DOI: 10.1055/s-0041-1735134
Review Article

Creating Clarity in Noisy Environments by Using Deep Learning in Hearing Aids

Asger Heidemann Andersen
1   Oticon A/S, Smørum, Denmark
,
Sébastien Santurette
1   Oticon A/S, Smørum, Denmark
,
Michael Syskind Pedersen
1   Oticon A/S, Smørum, Denmark
,
Emina Alickovic
2   Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
,
Lorenz Fiedler
2   Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
,
Jesper Jensen
1   Oticon A/S, Smørum, Denmark
,
Thomas Behrens
1   Oticon A/S, Smørum, Denmark
› Author Affiliations
› Further Information
   Permissions and Reprints

Abstract

Hearing aids continue to acquire increasingly sophisticated sound-processing features beyond basic amplification. On the one hand, these have the potential to add user benefit and allow for personalization. On the other hand, if such features are to benefit according to their potential, they require clinicians to be acquainted with both the underlying technologies and the specific fitting handles made available by the individual hearing aid manufacturers. Ensuring benefit from hearing aids in typical daily listening environments requires that the hearing aids handle sounds that interfere with communication, generically referred to as “noise.” With this aim, considerable efforts from both academia and industry have led to increasingly advanced algorithms that handle noise, typically using the principles of directional processing and postfiltering. This article provides an overview of the techniques used for noise reduction in modern hearing aids. First, classical techniques are covered as they are used in modern hearing aids. The discussion then shifts to how deep learning, a subfield of artificial intelligence, provides a radically different way of solving the noise problem. Finally, the results of several experiments are used to showcase the benefits of recent algorithmic advances in terms of signal-to-noise ratio, speech intelligibility, selective attention, and listening effort.

Keywords

noise reduction - beamforming - directionality - postfiltering


Publication History

Article published online:
24 September 2021

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA

 

  • References

  • 1 Kochkin S. MarkeTrak VIII: consumer satisfaction with hearing aids is slowly increasing. Hear J 2010; 63 (01) 19-32
    CrossrefPubMedGoogle Scholar
  • 2 Picou EM. MarkeTrak 10 (MT10) survey results demonstrate high satisfaction with and benefits from hearing aids. Semin Hear 2020; 41 (01) 21-36
    Article in Thieme ConnectPubMedGoogle Scholar
  • 3 Moore BCJ. Cochlear Hearing Loss: Physiological, Psychological and Technical Issues. 2nd ed.. Wiley; 2007
    CrossrefGoogle Scholar
  • 4 Plomp R. Auditory handicap of hearing impairment and the limited benefit of hearing aids. J Acoust Soc Am 1978; 63 (02) 533-549
    CrossrefPubMedGoogle Scholar
  • 5 Lopez RS, Bianchi F, Fereczkowski M, Santurette S, Dau T. Data-driven approach for auditory profiling. In: Proceedings of the International Symposium on Auditory and Audiological Research. Nyborg, Denmark: 2017: 247-254
    PubMedGoogle Scholar
  • 6 Moore B. An Introduction to the Psychology of Hearing. 6th ed.. Leiden, Netherlands: Brill; 2013
    Google Scholar
  • 7 Elko GW. Superdirectional microphone arrays. In: Gay SL, Benesty J. eds. Acoustic Signal Processing for Telecommunication. New York, United States: Springer; 2000: 181-237
    CrossrefGoogle Scholar
  • 8 Capon J. High-resolution frequency-wavenumber spectrum analysis. Proc IEEE 1969; 57 (08) 1408-1418
    CrossrefPubMedGoogle Scholar
  • 9 Cox H, Zeskind R, Owen M. Robust adaptive beamforming. IEEE Trans Acoust Speech Signal Process 1987; 35 (10) 1365-1376
    CrossrefPubMedGoogle Scholar
  • 10 Wiener N. Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications. Cambridge: MIT Press; 1949
    CrossrefGoogle Scholar
  • 11 Ephraim Y, Malah D. Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator. IEEE Trans Acoust Speech Signal Process 1984; 32 (06) 1109-1121
    CrossrefPubMedGoogle Scholar
  • 12 Gannot S, Vincent E, Markovich-Golan S, Ozerov A. A consolidated perspective on multimicrophone speech enhancement and source separation. IEEE/ACM Trans Acoust Speech Signal Process 2017; 25 (04) 692-730
    PubMedGoogle Scholar
  • 13 Kjems U, Boldt JB, Pedersen MS, Lunner T, Wang D. Role of mask pattern in intelligibility of ideal binary-masked noisy speech. J Acoust Soc Am 2009; 126 (03) 1415-1426
    CrossrefPubMedGoogle Scholar
  • 14 Doclo S. Multi-microphone noise reduction and dereverberation techniques for speech applications. PhD thesis. KU Leuven; Leuven, Belgium: 2003
    Google Scholar
  • 15 Simmer KU, Bitzer J, Marro C. Post-filtering techniques. In: Brandstein M, Ward D. eds. Microphone Arrays: Signal Processing Techniques and Applications. Springer; 2001: 39-60
    CrossrefGoogle Scholar
  • 16 Kjems U, Jensen J. Maximum likelihood based noise covariance matrix estimation for multi-microphone speech enhancement. In: Proceedings of the 20th European Signal Processing Conference (EUSIPCO). Bucharest, Romania 2012: 95-299
    PubMedGoogle Scholar
  • 17 Jensen J, Pedersen MS. Analysis of beamformer directed single-channel noise reduction system for hearing aid applications. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) South Brisbane, Queensland, Australia 2015: 5728-5732
    PubMedGoogle Scholar
  • 18 Boldt J, Kjems U, Pedersen MS, Lunner T, Wang D. Estimation of the ideal binary mask using directional systems. In: Proceedings of the 11th International Workshop on Acoustic Echo and Noise Control Seattle, Washington USA 2008
    PubMedGoogle Scholar
  • 19 Neher T, Wagener KC. Investigating differences in preferred noise reduction strength among hearing aid users. Trends Hear 2016; 20: 20
    PubMedGoogle Scholar
  • 20 Kim G, Loizou PC. Gain-induced speech distortions and the absence of intelligibility benefit with existing noise-reduction algorithms. J Acoust Soc Am 2011; 130 (03) 1581-1596
    CrossrefPubMedGoogle Scholar
  • 21 Dillon H. Hearing Aids. Thieme; 2000
    Google Scholar
  • 22 Kuklasiński A, Doclo S, Jensen SH, Jensen J. Maximum likelihood PSD estimation for speech enhancement in reverberation and noise. IEEE/ACM Trans Acoust Speech Signal Process 2016; 24 (09) 1599-1612
    PubMedGoogle Scholar
  • 23 Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016
    Google Scholar
  • 24 Bishop C. Pattern Recognition and Machine Learning. Springer; 2006
    Google Scholar
  • 25 Brown T, Mann B, Ryder N. et al. Language models are few-shot learners. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H. eds. Advances in Neural Information Processing Systems. 2020. 33. 1877-1901
    Google Scholar
  • 26 Kolbæk M, Tan ZH, Jensen J. Speech intelligibility potential of general and specialized deep neural network based speech enhancement systems. IEEE/ACM Trans Acoust Speech Signal Process 2017; 25 (01) 153-167
    PubMedGoogle Scholar
  • 27 Healy EW, Yoho SE, Wang Y, Wang D. An algorithm to improve speech recognition in noise for hearing-impaired listeners. J Acoust Soc Am 2013; 134 (04) 3029-3038
    CrossrefPubMedGoogle Scholar
  • 28 Healy EW, Yoho SE, Chen J, Wang Y, Wang D. An algorithm to increase speech intelligibility for hearing-impaired listeners in novel segments of the same noise type. J Acoust Soc Am 2015; 138 (03) 1660-1669
    CrossrefPubMedGoogle Scholar
  • 29 Chen J, Wang Y, Yoho SE, Wang D, Healy EW. Large-scale training to increase speech intelligibility for hearing-impaired listeners in novel noises. J Acoust Soc Am 2016; 139 (05) 2604-2612
    CrossrefPubMedGoogle Scholar
  • 30 Kim G, Lu Y, Hu Y, Loizou PC. An algorithm that improves speech intelligibility in noise for normal-hearing listeners. J Acoust Soc Am 2009; 126 (03) 1486-1494
    CrossrefPubMedGoogle Scholar
  • 31 Xu Y, Du J, Dai L, Lee C. An experimental study on speech enhancement based on deep neural networks. IEEE Signal Process Lett 2014; 21 (01) 65-68
    CrossrefPubMedGoogle Scholar
  • 32 Minnaar P, Albeck SF, Simonsen CS, Søndersted B, Oakley SAD, Bennedbæk J. Reproducing real-life listening situations in the laboratory for testing hearing aids. In: Audio Engineering Society Convention 135: Paper 8951, 2013
    PubMedGoogle Scholar
  • 33 Kirkeby O, Nelson PA, Hamada H, Orduna-Bustamante F. Fast deconvolution of multichannel systems using regularization. IEEE Trans Acoust Speech Signal Process 1998; 6 (02) 189-194
    CrossrefPubMedGoogle Scholar
  • 34 Brumm H, Zollinger S. The evolution of the Lombard effect: 100 years of psychoacoustic research. Behaviour 2011; 148 (11/13): 1173-1198
    CrossrefPubMedGoogle Scholar
  • 35 Berouti M, Schwartz R, Makhoul J. Enhancement of speech corrupted by acoustic noise. IEEE Trans Acoust Speech Signal Process 1979: 208-211
    PubMedGoogle Scholar
  • 36 Andersen AH, Jensen J, Pedersen MS. et al. Hearing device comprising a noise reduction system. United States Patent Application Publication No. US 2020/0260198 A1
    PubMedGoogle Scholar
  • 37 Hagerman B, Olofsson Å. A method to measure the effect of noise reduction algorithms using simultaneous speech and noise. Acta Acust United Acust 2004; 90 (02) 356-361
    PubMedGoogle Scholar
  • 38 Wardenga N, Batsoulis C, Wagener KC, Brand T, Lenarz T, Maier H. Do you hear the noise? The German matrix sentence test with a fixed noise level in subjects with normal hearing and hearing impairment. Int J Audiol 2015; 54 (Suppl. 02) 71-79
    CrossrefPubMedGoogle Scholar
  • 39 Buus S, Florentine M. Growth of loudness in listeners with cochlear hearing losses: recruitment reconsidered. J Assoc Res Otolaryngol 2002; 3 (02) 120-139
    CrossrefPubMedGoogle Scholar
  • 40 Ohlenforst B, Zekveld AA, Jansma EP. et al. Effects of hearing impairment and hearing aid amplification on listening effort: a systematic review. Ear Hear 2017; 38 (03) 267-281
    CrossrefPubMedGoogle Scholar
  • 41 Ohlenforst B, Wendt D, Kramer SE, Naylor G, Zekveld AA, Lunner T. Impact of SNR, masker type and noise reduction processing on sentence recognition performance and listening effort as indicated by the pupil dilation response. Hear Res 2018; 365: 90-99
    CrossrefPubMedGoogle Scholar
  • 42 Alickovic E, Lunner T, Wendt D. et al. Neural representation enhanced for speech and reduced for background noise with a hearing aid noise reduction scheme during a selective attention task. Front Neurosci 2020; 14: 846
    CrossrefPubMedGoogle Scholar
  • 43 Lunner T, Alickovic E, Graversen C, Ng EHN, Wendt D, Keidser G. three new outcome measures that tap into cognitive processes required for real-life communication. Ear Hear 2020; 41 (Suppl. 01) 39S-47S
    CrossrefPubMedGoogle Scholar
  • 44 Alickovic E, Ng EHN, Fiedler L, Santurette S, Innes-Brown H, Graversen C. Effects of hearing aid noise reduction on early and late cortical representations of competing talkers in noise. Front Neurosci 2021; 15: 636060
    CrossrefPubMedGoogle Scholar
  • 45 Fiedler L, Seifi Ala T, Graversen C, Alickovic E, Lunner T, Wendt D. Hearing Aid Noise Reduction Lowers the Sustained Listening Effort During Continuous Speech in Noise—A Combined Pupillometry and EEG Study. Ear and Hearing ; in press DOI: 10.1097/AUD.0000000000001050.
    CrossrefPubMedGoogle Scholar
  • 46 Fiedler L, Wöstmann M, Herbst SK, Obleser J. Late cortical tracking of ignored speech facilitates neural selectivity in acoustically challenging conditions. Neuroimage 2019; 186: 33-42
    CrossrefPubMedGoogle Scholar
  • 47 O'Sullivan J, Herrero J, Smith E. et al. Hierarchical encoding of attended auditory objects in multi-talker speech perception. Neuron 2019; 104 (06) 1195-1209.e3
    CrossrefPubMedGoogle Scholar
  • 48 Zion Golumbic EM, Ding N, Bickel S. et al. Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”. Neuron 2013; 77 (05) 980-991
    CrossrefPubMedGoogle Scholar
  • 49 Mesgarani N, Chang EF. Selective cortical representation of attended speaker in multi-talker speech perception. Nature 2012; 485 (7397): 233-236
    CrossrefPubMedGoogle Scholar
  • 50 Seifi Ala T, Graversen C, Wendt D, Alickovic E, Whitmer WM, Lunner T. An exploratory study of EEG alpha oscillation and pupil dilation in hearing-aid users during effortful listening to continuous speech. PLoS One 2020; 15 (07) e0235782
    CrossrefPubMedGoogle Scholar
  • 51 Pichora-Fuller MK, Kramer SE, Eckert MA. et al. Hearing impairment and cognitive energy: the framework for understanding effortful listening (FUEL). Ear Hear 2016; 37 (Suppl. 01) 5S-27S
    CrossrefPubMedGoogle Scholar