Why Ecological Momentary Assessment Surveys Go Incomplete?: When It Happens and How It Impacts DataFunding The present study was supported by the grants from Starkey Hearing Technologies, the National Institute on Deafness and Other Communication Disorders (R01DC015997), the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR 90REGE0013), and National Science Foundation (SCH 1838830).
Background Ecological momentary assessment (EMA) often requires respondents to complete surveys in the moment to report real-time experiences. Because EMA may seem disruptive or intrusive, respondents may not complete surveys as directed in certain circumstances.
Purpose This article aims to determine the effect of environmental characteristics on the likelihood of instances where respondents do not complete EMA surveys (referred to as survey incompletion), and to estimate the impact of survey incompletion on EMA self-report data.
Research Design An observational study.
Study Sample Ten adults hearing aid (HA) users.
Data Collection and Analysis Experienced, bilateral HA users were recruited and fit with study HAs. The study HAs were equipped with real-time data loggers, an algorithm that logged the data generated by HAs (e.g., overall sound level, environment classification, and feature status including microphone mode and amount of gain reduction). The study HAs were also connected via Bluetooth to a smartphone app, which collected the real-time data logging data as well as presented the participants with EMA surveys about their listening environments and experiences. The participants were sent out to wear the HAs and complete surveys for 1 week. Real-time data logging was triggered when participants completed surveys and when participants ignored or snoozed surveys. Data logging data were used to estimate the effect of environmental characteristics on the likelihood of survey incompletion, and to predict participants' responses to survey questions in the instances of survey incompletion.
Results Across the 10 participants, 715 surveys were completed and survey incompletion occurred 228 times. Mixed effects logistic regression models indicated that survey incompletion was more likely to happen in the environments that were less quiet and contained more speech, noise, and machine sounds, and in the environments wherein directional microphones and noise reduction algorithms were enabled. The results of survey response prediction further indicated that the participants could have reported more challenging environments and more listening difficulty in the instances of survey incompletion. However, the difference in the distribution of survey responses between the observed responses and the combined observed and predicted responses was small.
Conclusion The present study indicates that EMA survey incompletion occurs systematically. Although survey incompletion could bias EMA self-report data, the impact is likely to be small.
Received: 21 April 2020
Accepted: 01 June 2020
15 December 2020 (online)
© 2020. American Academy of Audiology. This article is published by Thieme.
Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA
- 1 Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol 2008; 4: 1-32
- 2 Timmer BHB, Hickson L, Launer S. Ecological momentary assessment: feasibility, construct validity, and future applications. Am J Audiol 2017; 26 (3S): 436-442
- 3 Wu YH, Stangl E, Zhang X, Bentler RA. Construct validity of the ecological momentary assessment in audiology research. J Am Acad Audiol 2015; 26 (10) 872-884
- 4 Wu YH, Stangl E, Chipara O, Zhang X. Test-retest reliability of ecological momentary assessment in audiology research. J Am Acad Audiol, In press
- 5 Wu YH, Stangl E, Chipara O, Gudjonsdottir A, Oleson J, Bentler RA. Comparison of in-situ and retrospective self-reports on assessing hearing aid outcomes. J Am Acad Audiol, In press
- 6 Preminger JE, Cunningham DR. Case-study analysis of various field study measures. J Am Acad Audiol 2003; 14 (01) 39-55
- 7 Walden BE, Surr RK, Cord MT, Dyrlund O. Predicting hearing aid microphone preference in everyday listening. J Am Acad Audiol 2004; 15 (05) 365-396
- 8 Wu YH, Bentler RA. Impact of visual cues on directional benefit and preference: part II--field tests. Ear Hear 2010; 31 (01) 35-46
- 9 Wu YH, Bentler RA. Do older adults have social lifestyles that place fewer demands on hearing?. J Am Acad Audiol 2012; 23 (09) 697-711
- 10 Bentler R, Wu YH, Kettel J, Hurtig R. Digital noise reduction: outcomes from laboratory and field studies. Int J Audiol 2008; 47 (08) 447-460
- 11 Galvez G, Turbin MB, Thielman EJ, Istvan JA, Andrews JA, Henry JA. Feasibility of ecological momentary assessment of hearing difficulties encountered by hearing aid users. Ear Hear 2012; 33 (04) 497-507
- 12 Timmer BHB, Hickson L, Launer S. Do hearing aids address real-world hearing difficulties for adults with mild hearing impairment? Results from a pilot study using ecological momentary assessment. Trends Hear 2018; 22: 2331216518783608
- 13 Wu YH, Stangl E, Chipara O, Hasan SS, DeVries S, Oleson J. Efficacy and effectiveness of advanced hearing aid directional and noise reduction technologies for older adults with mild to moderate hearing loss. Ear Hear 2019; 40 (04) 805-822
- 14 Henry JA, Galvez G, Turbin MB, Thielman EJ, McMillan GP, Istvan JA. Pilot study to evaluate ecological momentary assessment of tinnitus. Ear Hear 2012; 33 (02) 179-290
- 15 Pentony S, Xu J, Wu YH, Galster J. Using ecological momentary assessment in audiology research: the participants' perspective. Paper presented at the American Auditory Society, Scottsdale, AZ; 2018. Available at: https://haar.lab.uiowa.edu/sites/haar.lab.uiowa.edu/files/wysiwyg_uploads/pentony_et_al_aas_2018_2010.pdf
- 16 Shiffman S. Designing protocols for ecological momentary assessment. In: Stone A, Shiffman S, Atienza A, Nebeling L. eds. The Science of Real-Time Data Capture: Self-Reports in Health Research. New York, NY: Oxford University Press; 2007: 27-53
- 17 Ricketts TA, Bentler RA, Mueller HG. Pre-fitting tests using frequency-specific measures. In: Essentials of Modern Hearing Aids: Selection, Fitting, and Verification. San Diego, CA: Plural Publishing; 2018: 101-134
- 18 Humes LE, Rogers SE, Main AK, Kinney DL. The acoustic environments in which older adults wear their hearing aids: insights from datalogging sound environment classification. Am J Audiol 2018; 27 (04) 594-603