The Collaboration between Hearing Aid Users and Artificial Intelligence to Optimize Sound

Abstract

Hearing aid gain and signal processing are based on assumptions about the average user in the average listening environment, but problems may arise when the individual hearing aid user differs from these assumptions in general or specific ways. This article describes how an artificial intelligence (AI) mechanism that operates continuously on input from the user may alleviate such problems by using a type of machine learning known as Bayesian optimization. The basic AI mechanism is described, and studies showing its effects both in the laboratory and in the field are summarized. A crucial fact about the use of this AI is that it generates large amounts of user data that serve as input for scientific understanding as well as for the development of hearing aids and hearing care. Analyses of users' listening environments based on these data show the distribution of activities and intentions in situations where hearing is challenging. Finally, this article demonstrates how further AI-based analyses of the data can drive development.

Keywords

hearing aid classification - artificial intelligence - listening intention - data-driven hearing care

Publication History

Article published online:
24 September 2021

• References

• 1 Ferguson MA, Kitterick PT, Chong LY, Edmondson-Jones M, Barker F, Hoare DJ. Hearing aids for mild to moderate hearing loss in adults. (Review) Cochrane Database Syst Rev 2017; 9 (09) CD012023
• 2 Keidser G, Dillon H. What's new in prescriptive fittings down under?. In: Palmer CV, Seewald R. (eds.). Hearing Care for Adults. Stäfa, Switzerland: Phonak AG; 2006
• 3 Scollie S, Seewald R, Cornelisse L. et al. The desired sensation level multistage input/output algorithm. Trends Amplif 2005; 9 (04) 159-197
• 4 Keidser G, Dillon H, Flax M, Ching T, Brewer S. The NAL-NL2 prescription procedure. Audiology Res 2011; 1 (1, 1S) e24
• 5 Smeds K, Keidser G, Zakis J. et al. Preferred overall loudness. II: Listening through hearing aids in field and laboratory tests. Int J Audiol 2006; 45 (01) 12-25
• 6 Smeds K, Keidser G, Zakis J. et al. Preferred overall loudness. I: Sound field presentation in the laboratory. Int J Audiol 2006; 45 (01) 2-11
• 7 Bentler RA, Cooley LJ. An examination of several characteristics that affect the prediction of OSPL90 in hearing aids. Ear Hear 2001; 22 (01) 58-64
• 8 Caswell-Midwinter B, Whitmer WM. The perceptual limitations of troubleshooting hearing-aids based on patients' descriptions. Int J Audiol 2021; 60 (06) 427-437
• 9 Kuk FK. Paired comparisons as a fine-tuning tool in hearing aid fittings. In: Valente M. (ed.). Strategies for Selecting and Verifying Hearing Aid Fittings. New York: Thieme Medical Publishing; 1992: 125-150
• 10 Kuk FK, Lau C. Comparison of preferred frequency gain settings obtained with category rating and modified simplex procedure. J Am Acad Audiol 1996; 7 (05) 322-331
• 11 Amlani AM, Schafer EC. Application of paired-comparison methods to hearing AIDS. Trends Amplif 2009; 13 (04) 241-259
• 12 Rasmussen CE, Williams CKI. Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press; 2006
• 13 MacKay DJC. Information Theory, Inference and Learning Algorithms. Cambridge, UK: Cambridge University Press; 2003
• 14 Mockus J. On Bayesian methods for seeking the extremum. In: Proceedings of the IFIP Technical Conference; July 1974: 400-404
• 15 Jones DR, Schonlau M, Welch WJ. Efficient global optimization of expensive black-box functions. J Glob Optim 1998; 13 (04) 455-492
• 16 Shahriari B, Swersky K, Wang Z, Adams RP, De Freitas N. Taking the human out of the loop: a review of Bayesian optimization. Proc IEEE 2015; 104 (01) 148-175
• 17 Nielsen JBB, Nielsen J, Larsen J. Perception-based personalization of hearing aids using gaussian processes and active learning. IEEE/ACM Trans Audio Speech Lang Process 2014; 23 (01) 162-173
• 18 Jensen NS, Hau O, Nielsen JBB, Nielsen TB, Legarth SV. Perceptual Effects of Adjusting Hearing-Aid Gain by Means of a Machine-Learning Approach Based on Individual User Preference. Trends Hear 2019; 23: 1-23
• 19 Townend O, Nielsen JB, Balslev D. SoundSense Learn—listening intention and machine learning. Hear Rev 2018; 25 (06) 28-31
• 20 Balling LW, Townend O, Switalski W. Real-life hearing aid benefit with Widex EVOKE. Hear Rev 2019; 26 (03) 30-36
• 21 Jensen NS, Hau O, Lelic D, Herrlin P, Wolters F, Smeds K. Evaluation of auditory reality and hearing aids using an ecological momentary assessment (EMA) approach. Proc Int Congr Acoust 2019; 2019 (September): 6545-6552
• 22 Schinkel-Bielefeld N, Kunz P, Zutz A, Buder B. Evaluation of hearing aids in everyday life using ecological momentary assessment: What situations are we missing?. Am J Audiol 2020; 29 (3S): 591-609
• 23 Holube I, von Gablenz P, Kowalk U, Bitzer J. Assessment of acoustical properties and subjective perception in everyday life. In: Proceedings of the 23rd International Congress on Acoustics, Aachen; 2019: 7639
• 24 Balling LW, Townend O, Mølgaard LL, Jespersen CB, Switalski W. AI-driven insights from AI-driven data. Hear Rev 2021; 28 (01) 27-29
• 25 The Nielsen Total Audience Report. August 2020. Available from https://www.nielsen.com/us/en/insights/report/2020/the-nielsen-total-audience-report-august-2020/, last accessed August 9, 2021
• 26 Picou EM. MarkeTrak 10 (MT10) survey results demonstrate high satisfaction with and benefits from hearing aids. Semin Hear 2020; 41 (01) 21-36
• 27 Bronkhorst AW. The cocktail party phenomenon: a review of research on speech intelligibility in multiple-talker conditions. Acta Acust 2000; 86 (01) 117/128
• 28 Nielsen JBB, Mølgaard LL, Jensen NS, Balling LW. Machine-learning based hearing-aid personalization: preference optimizer, preference sensor or both?. In: 7th International Symposium on Auditory and Audiological Research. 2019: 40
• 29 Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 2002; 2002: 603-619
• 30 Brændgaard M. An Introduction to MoreSound Intelligence. Tech Paper 2020. . Available from https://wdh01.azureedge.net/-/media/oticon/main/pdf/master/whitepaper/69674uk_tech_paper_moresound_intelligence.pdf?la=en&rev=3F19&hash=6A1037B3951F262345E45C4A725D3CC7, last accessed August 9, 2021
• 31 Boothroyd A, Mackersie C. A “goldilocks” approach to hearing-aid self-fitting: user interactions. Am J Audiol 2017; 26 (3S): 430-435
• 32 Mackersie C, Boothroyd A, Lithgow A. A “Goldilocks” approach to hearing aid self-fitting: ear-canal output and speech intelligibility index. Ear Hear 2019; 40 (01) 107-115
• 33 Nelson PB, Perry TT, Gregan M, VanTasell D. Self-adjusted amplification parameters produce large between-subject variability and preserve speech intelligibility. Trends Hear 2018; 22: 2331216518798264