The Effect of Learning Modality and Auditory Feedback on Word Memory: Cochlear-Implanted versus Normal-Hearing Adults

Abstract

Background:

In recent years, the effect of cognitive abilities on the achievements of cochlear implant (CI) users has been evaluated. Some studies have suggested that gaps between CI users and normal-hearing (NH) peers in cognitive tasks are modality specific, and occur only in auditory tasks.

Purpose:

The present study focused on the effect of learning modality (auditory, visual) and auditory feedback on word memory in young adults who were prelingually deafened and received CIs before the age of 5 yr, and their NH peers.

Research Design:

A production effect (PE) paradigm was used, in which participants learned familiar study words by vocal production (saying aloud) or by no-production (silent reading or listening). Words were presented (1) in the visual modality (written) and (2) in the auditory modality (heard). CI users performed the visual condition twice—once with the implant ON and once with it OFF. All conditions were followed by free recall tests.

Study Sample:

Twelve young adults, long-term CI users, implanted between ages 1.7 and 4.5 yr, and who showed ≥50% in monosyllabic consonant-vowel-consonant open-set test with their implants were enrolled. A group of 14 age-matched NH young adults served as the comparison group.

Data Collection and Analysis:

For each condition, we calculated the proportion of study words recalled. Mixed-measures analysis of variances were carried out with group (NH, CI) as a between-subjects variable, and learning condition (aloud or silent reading) as a within-subject variable. Following this, paired sample t tests were used to evaluate the PE size (differences between aloud and silent words) and overall recall ratios (aloud and silent words combined) in each of the learning conditions.

Results:

With visual word presentation, young adults with CIs (regardless of implant status CI-ON or CI-OFF), showed comparable memory performance (and a similar PE) to NH peers. However, with auditory presentation, young adults with CIs showed poorer memory for nonproduced words (hence a larger PE) relative to their NH peers.

Conclusions:

The results support the construct that young adults with CIs will benefit more from learning via the visual modality (reading), rather than the auditory modality (listening). Importantly, vocal production can largely improve auditory word memory, especially for the CI group.

Key Words

auditory feedback - cochlear implants - memory - production effect - study modality

INTRODUCTION

Cochlear implants (CIs) are now standard of care for the severe-to-profound hearing-impaired individuals who do not gain enough benefit from high-power digital hearing aids ([Wilson, 2015]). The great bulk of outcome studies have documented the success of CIs in both children and adults ([De Raeve et al, 2015]; [Moberly et al, 2016]). Yet, the perceptual learning abilities of people with CIs are not fully understood. Specifically, it is not clear how young adults, who were prelingually deafened and implanted on early childhood, perform on long-term word memory tasks compared to normal-hearing (NH) individuals. Does this group rely mainly on visual routes to learning and language access, or on auditory pathways? Obviously, implanted children and adults, characterized with different levels of auditory abilities and different cognitive skills, may use diverse encoding, storage, and processing strategies ([Pisoni, 2000]). These strategies may or may not be similar to those of hearing individuals. Following this line of reasoning, the present study evaluated visual and auditory long-term word memory, comparing the performance of young adult CI users and their NH peers. Understanding long-term memory abilities and the impact of study modality can improve intervention and (re)habilitation programs for CI users.

CI individuals show a wide range of speech perception, speech production, and language abilities ([Svirsky et al, 2000]; [Taitelbaum-Swead et al, 2005]; [Tait et al, 2010]). Such high variability is also prevalent in prelingually implanted young adults. This subgroup was of particular interest in the present study. The large variability observed in speech-language outcomes can be partially explained by background variables, such as physical-anatomic, demographic, social, and environmental factors ([Fryauf-Bertschy et al, 1997]; [Kirk et al, 2000]; [Niparko et al, 2010]; [Geers and Sedey, 2011]). Yet, there is a considerable amount of individual variability that cannot be explained by the aforementioned variables. This has led researchers to focus on neurocognitive and information-processing factors that may contribute to the variability in benefits one can achieve from the CI ([Pisoni and Geers, 2000]; [Cleary et al, 2001]; [Pisoni et al, 2011]).

COGNITIVE SKILLS IN PEDIATRIC CI USERS

From the pediatric perspective, atypical auditory experience (e.g., underspecified cognitive representations of degraded acoustic–phonetic input signals) as well as limitations in auditory input from a CI may affect neurocognitive development ([Kronenberger et al, 2011]). They may have adverse effects in the development of basic cognitive and executive functions: attention, memory, and learning ([Figueras et al, 2008]; [Pisoni et al, 2008]; [2010]).

Large individual differences in spoken word recognition skills and language development are observed in pediatric CI users. At least some of this variance may be attributed to cognitive factors related to the efficiency with which representations of spoken words are stored and retrieved from memory ([Pisoni, 2000]). On average, CI children score below age norms on many cognitive measures, such as speech perception performance ([Dawson et al, 2002]), verbal rehearsal speed ([Pisoni and Cleary, 2003]), and visual sequence memory and learning ([Cleary et al, 2001]; [Pisoni et al, 2008]).

Studies of memory abilities of children with CI have mainly been concerned with the role of working memory, a short-term memory buffer with a limited capacity, which allows for the processing and manipulation of stored information ([Baddeley and Hitch, 1998]). Those studies demonstrated that children with CI have shorter “auditory” working memory spans than their NH age-matched peers ([Pisoni and Geers, 2000]; [Pisoni and Cleary, 2003]). In addition, children with CIs fall below average compared with NH controls in their “visual” memory spans and some visual sequencing skills ([Cleary et al, 2001]; [Pisoni and Cleary, 2004]; [Conway et al, 2009]; [Engel-Yeger et al, 2011]).

Very few studies have investigated long-term memory (which stores information that was transferred from the working memory buffer, for long periods) of children with CIs ([Engel-Yeger et al, 2011]). Since children with CIs are at risk for compromised development of working memory skills ([Kronenberger et al, 2011]), long-term memory abilities may be negatively affected as well. This, in turn, may adversely affect other cognitive abilities (e.g., learning), and may have delayed effects in adolescence and adulthood as well.

COGNITIVE ABILITIES OF PRELINGUALLY DEAFENED ADULT CI USERS

Evaluating the cognitive skills of prelingually deafened adults who were implanted as young children raises some interesting questions. These CI users have years of auditory experience with the CI, along with long rehabilitation (mainly, participation in speech and language intervention programs). Through the intensive intervention program that usually follows an implant surgery, many CI users eventually develop open-set speech perception skills and are able to produce intelligible speech ([Geers et al, 2000]; [Cleary et al, 2001]). However, do they achieve similar performance to NH individuals in high-order cognitive tasks, or do the deficits typically observed in childhood persist? Can years of auditory practice close the gaps typically observed for children with CI?

Data regarding the cognitive abilities of adult prelingually deafened CI users who were implanted on childhood is scarce. [Pisoni et al (2011)] found significant impairment on auditory digit span measures for such CI users tested in high school (see also, [Geers et al, 2013]). Interestingly, teenage CI user performance in a reading span task (therefore a visual modality) was comparable to that obtained in a control group of NH teenagers ([Geers et al, 2013]). In fact, many of the abovementioned studies stressed modality-based differences in cognitive performance of hearing-impaired individuals and CI users ([Daneman and Carpenter, 1980]). Not surprisingly, auditory deprivation may lead to enhanced reliance on the visual modality, possibly as a compensatory mechanism ([Bavelier et al, 2006]). So, are memory deficits observed for CI users modality specific? To answer this question, a direct comparison of auditory and visual conditions is needed; such comparison could confirm whether memory performance differs across modalities in adult CI users.

With the aim of gaining a better understanding of “how” people acquire new information through a CI, and “what” are they learning, dedicated experimental methods are needed. In the field of memory research, one of these techniques is the production effect (PE) paradigm, which holds an advantage by allowing the pitting of visual and auditory modalities against each other. Importantly, this paradigm involves a long-term memory task, whereas most of the literature regarding CI listeners’ memory abilities has been focused on working (short-term) memory.

THE PE IN MEMORY

A familiar long-term memory phenomenon, the PE, refers to the memory advantage for words read aloud, relative to words read silently ([MacLeod et al, 2010]). In a typical PE experiment, a list of study words is visually presented: half of the words are learned by silent reading (silent words), and half by reading aloud (aloud words); a memory test is then given. Memory performance for aloud words has been found superior to that of silent words, both in recognition tests ([MacLeod, 2011]; [Forrin et al, 2012]; [Ozubko et al, 2012]) as well as free recall tests ([Lin and MacLeod, 2012]; [Castel et al, 2013]; [Icht et al, 2014]).

This simple but robust encoding technique was validated across numerous studies. Vocal production was found to improve memory relative to other types of unique production, such as whispering, typing, and writing ([Forrin et al, 2012]). Advantage for (vocal) production was also found when participants were prevented from auditory feedback. [Mama et al (submitted for publication)] used a visual PE paradigm with an auditory background noise that accompanied the study phase. Although the noise masked the participants’ own voices, a PE was still obtained.

Vocalization was found to enhance memory for various types of materials, such as pictures ([Icht and Mama, 2015]), nonwords ([MacLeod et al, 2010], Expt 6), sentences and text ([Ozubko et al, 2012]), and across different populations (e.g., older adults, [Lin and MacLeod, 2012]; children, [Icht and Mama, 2015]). Recently, the PE was expanded to the auditory modality, which involves the learning of aurally presented words (rather than written words; [Mama and Icht, 2016a]).

The prevailing theory of the PE is encoding distinctiveness—“at the time of test, a word that was produced at study has an additional source of discrimination relative to a word that was not produced” ([MacLeod et al, 2010], p. 681). According to this account, the produced words are more distinct relative to the backdrop of the silent (nonproduced) words. During presentation, memory traces are being created for each of the study words. The memory traces of the visually presented aloud words involve visual, motor, and auditory representations, thus are richer and deeper. The nonproduced words, on the other hand, are characterized by a sole visual (or auditory, with the aural presentation) trace, hence are weaker and shallower. The production that involves the greater number of unique encoding processes leads to the best memory ([Mama and Icht, 2016b]). Producing a word results in a quantitative as well as qualitatively different memory record than no-production, such as silent reading or listening ([Putnam et al, 2014]; [Mama and Icht, 2016a]); at retrieval, participants are able to use this distinctive information as part of their decision process to guide test performance ([Fawcett et al, 2012]) in favor of the produced words.

As the PE is considered a robust and reliable phenomenon, it was offered as a simple and effective mnemonic ([Lin and MacLeod, 2012]; [Icht and Mama, 2015]). In the present study, our main interest was gaining a better understanding of, and practical insights into, memory and learning among CI users. Considering their impaired auditory abilities, will vocal production improve visual word memory in this special population? Will vocalizing enhance memory for aurally presented words as well? Finally, what is the role of auditory feedback (hearing oneself vocalizing a visually presented study word) in CI user learning?

THE CURRENT STUDY

This study focused on auditory and visual long-term memory of young adults with CIs that were implanted in early childhood (all of them prelingually). The effect of learning modality (auditory, visual) was evaluated among this group and NH peers. In addition, the impact of auditory feedback (CI-ON versus CI-OFF) on visual word memory of CI users was tested. A PE paradigm was used, in which lists of study words were presented in the (a) visual modality (written words) or (b) auditory modality (heard words). The participants learned these study words, half by vocal production (saying the words aloud) and the remaining half by no-production (silent reading or listening). The CI users performed the visual condition twice, with the implant ON (CI-ON condition) and OFF (CI-OFF condition). Following each study phase (visual, auditory), a free recall test was performed.

This procedure allowed exploration of three major questions. First, whether vocal production improves memory for visually presented words in adults with CIs. We hypothesized a similar memory advantage would occur for CI users as the NH group (for related results, see [Geers et al, 2013]). Our main rationale was prior evidence that a PE (with visual study modality) can be obtained even with no auditory feedback (e.g., mouthing, [Forrin et al, 2012]; background noise, [Mama et al, submitted for publication]), let alone with a limited one, as that of the CI users.

The second goal of this study was to evaluate the relative impact of the implant’s ON/OFF status on visual learning (written words). Obviously, the CI improves auditory functions, hence it may improve general performance in cognitive (memory and learning) tasks. However, we wondered whether the CI-ON condition would result in higher recall rates than with the CI-OFF condition, with visual presentation. Answering this question carries clinical implications for the necessity of self-auditory feedback during speech (vocal production) to improve learning.

Finally, we explored the question of whether the magnitude of the PE using an auditory modality would be comparable among CI users and NH population. A recent PE study that tested NH listeners using an auditory presentation of study words found an advantage for vocalized words over nonproduced (heard only) words (yet relatively small in size; [Mama and Icht, 2016a]). Therefore, we expected a PE in CI users as well, but since the auditory modality is their inferior one, we predicted lower recall rates relative to NH participants ([Pisoni et al, 2011]).

METHOD

Participants

Two groups of participants were enrolled in the study: individuals with CIs and those with NH. All participants gave their informed consent to take part in the study, which was approved by the local ethics committee.

The CI group consisted of 12 implanted young adults who met the following inclusion criteria: (a) onset of severe-to-profound hearing impairment before 3 yr of age, (b) age at implantation before 5 yr, (c) hearing aid usage before implantation, (d) no other developmental or neurocognitive impairments, (e) mainstream education and oral communication, and (f) usage of multichannel CIs, (g) no less than 50% in monosyllabic consonant-vowel-consonant open-set test (Hebrew Arthur Boothroyd Test; [Boothroyd, 1968]).

CI Background Information

The mean chronological age of the implanted participants was 20 yr (standard deviation [SD] = 4.2). Nine of the participants used two implants, while three participants used one implant with no hearing aid in the second ear. Mean age at implantation (first implant for nine participants) was 2.8 yr (SD = 0.93), and mean age of second implantation was 13 yr (SD = 1.95). Etiology for hearing loss was genetic for eight participants and unknown for four participants. All participants used a multichannel Nucleus device (Cochlear TM, different generations) ([Table 1]).

Table 1 Individual Background Data of the CI Group

SI No.	Gender	Chronological Age (Yr, Mo)	Age at First Implantation (Yr, Mo)	Age at Second Implantation (Yr, Mo)	Etiology
1	M	22	3, 9	14	Genetic
2	F	16, 6	3	11, 6	Unknown
3	F	17	2	11	Unknown
4	M	25	4, 6	—	Genetic
5	M	14	1, 7	—	Genetic
6	F	24, 6	3	14	Genetic
7	F	24, 6	3	14	Genetic
8	F	14, 6	1, 6	10	Genetic
9	F	22	3	16, 6	Unknown
10	F	17	2, 6	13	Genetic
11	F	17	2, 6	13	Genetic
12	F	24	4	—	Unknown

Note: F = female; M = male; SI No. = serial number.

The NH group consisted of 14 young men and women, undergraduate students from Ariel University, who met the following inclusion criteria: (a) were native Hebrew speakers, (b) reported no developmental and cognitive impairments, and (c) had normal air-conduction hearing thresholds <15 dB HL bilaterally at octave frequencies from 0.25 to 4 kHz.

Apparatus and Stimuli

The pool of study words consisted of 90 Hebrew words, bisyllabic nouns, three to five letters long, with frequencies of >12 per million (taken from [Icht et al, 2014]). For the auditory presentation, 30 words were randomly selected from this pool. These 30 words were announced by a female speaker and recorded in a professional radio studio using the Samplitude classic 8.1 program (MAGIX AG) and a TCS-6 microphone (Sontronics). During the study phase, each of the 30 study words was aurally presented via personal computer loudspeakers, at 70 dB SPL, under the control of PowerPoint program.

For the visual presentation, the remaining 60 words were used, 30 different words for the each visual condition (implant-ON, implant-OFF). In these conditions, each word was presented singly for view. The word appeared at the center of a 15″ color monitor (Compaq [Houston, TX] laptop computer under control of PowerPoint program). The words were presented in black (28-point Arial), against a white background.

On each trial, either auditory or visual, a small icon (2 cm²) appeared ∼5 cm above the center of the screen (which was otherwise blank for the auditory condition or presenting the written study word for the visual conditions). The icon entailed a small picture of closed lips or of a microphone. The icon indicated the appropriate mode of production for that word: the lips indicated silent reading or listening, the microphone pointed to vocal production.

Design

Before the experiment, implanted participants were tested on the Hebrew Arthur Boothroyd open-set meaningful word test. Participants heard 2 lists of 10 words each (in a random order from a pool of 15 lists). The words were presented via laptop computer in 70 dB SPL. NH participants performed a hearing screening test, using an AD229B audiometer (Interacoustics).

Study

For the CI group, we used three learning conditions (e.g., three separate experimental blocks): (a) visual-CI-ON: visual word presentation with the implant ON; (b) visual-CI-OFF: visual word presentation with the implant OFF; and (c) auditory-CI-ON: auditory presentation with the implant ON. For the NH group, we used two learning conditions: (a) visual presentation and (b) auditory presentation. The order of the blocks was random across participants.

In each of these learning conditions that were sequentially performed, 30 study words were randomly selected for presentation (visually or aurally); for 15 words, the requested mode of learning was vocal production (vocalized, or aloud words), and for the remaining 15 words, the mode of learning was no-production (silent reading in the visual conditions, listening in the auditory condition).

Each experimental block started with a visual presentation of the icon (lips or microphone). The study word was aurally or visually presented 300 msec following the icon’s appearance (the visual word appeared for 1 sec). After the item’s presentation, the icon remained visible for 3,000 msec. A blank screen for 1 sec followed (thus the interval between words was ∼4 sec). A ten-minute break was given between each block.

Filler Task

Four arithmetic problems (multiplication of four-digit numbers) printed on an A4 paper were prepared for filler tasks. Following each study phase, participants were given 4 min to complete these problems.

Memory Test—Free Recall

Following each study phase and filler task, a free recall test was performed (a total of three tests for CI, and two tests for NH). Each participant was asked to write down from memory as many study words as she or he could recall. An empty sheet of paper and a pencil were provided by the experimenter (free recall was performed with no time limit).

Procedure

The participants were tested individually in a quiet room. Upon arrival, each participant read and signed the informed consent form, and was seated at a distance of 60 cm from the center of the computer screen. The participant was told that the goal was to learn each word via the mode signaled by the icon (lips or microphone) and that memory test would follow the presentation of the words.

For the auditory presentation, the study words were played from two loudspeakers (Electro-Medical Instrument Co., Mississauga, ON, Canada), each positioned at the height of the listener’s head at 45° azimuth. For the visual presentation, the study words were presented on the computer screen. A research assistant accompanied each participant throughout the experimental session, ascertaining the accurate vocal production of the study words (note that no errors occurred for both presentation modalities).

After the presentation of the first block (30 words), the participants performed the short filler task. Free recall test followed, performed by writing down as many study words from memory as possible. Following this, a ten-minute break was given, and the second block begun (study phase, filler task, and test). Since the CI group performed the visual learning twice (visual-CI-ON and visual-CI-OFF), a third experimental block followed in a similar fashion. The whole experimental session lasted ≤50 min.

RESULTS

Visual Modality PE: NH and CI with the Implant ON versus OFF

[Figure 1] gives the results of free recall tests for the visual conditions among the CI (visual-CI-ON, visual-CI-OFF) and NH groups. Plotted are the proportions of correctly recalled words (aloud and silent). Visual inspection reveals the superiority of vocal production (aloud words) over silent reading (silent words), therefore demonstrating a PE, across all learning conditions. As can be seen in [Figure 1] (left side), for the NH group, a significantly higher recall rate was found for aloud words (Mean [M] = 0.27, SD = 0.13) relative to silent words (M = 0.16, SD = 0.09). A similar pattern was observed for the visual-CI-ON condition ([Figure 1], middle), with higher recall rates for aloud words (M = 0.32, SD = 0.12) than for silent words (M = 0.2, SD = 0.15). A comparable pattern was observed for the visual-CI-OFF condition ([Figure 1], right), with higher recall rates for aloud words (M = 0.28, SD = 0.12) than for silent words (M = 0.16, SD = 0.11).

Figure 1 Percentage of recalled words in the “visual” learning conditions, calculated separately for the subsets of silent and aloud words, for CI users (visual-CI-ON and visual-CI-OFF) and NH group. The error bars are standard errors of their respective means.

A repeated measures analysis of variance (ANOVA) with learning condition (aloud words, silent words) as a within-subjects variable, and group (NH, CI-ON, CI-OFF) as a between-subjects variable, revealed a significant main effect for learning condition [F _(1,36) = 39.07, p < 0.0001, η² = 0.52). No group effect was found [F _(2,36) = 0.59, p > 0.05], and no learning condition × group interaction was found [F _(2,36) = 0.04, p > 0.05].

Auditory Modality PE: NH and CI

[Figure 2] gives the results of free recall tests for the auditory condition. Plotted are the proportions of correctly recalled words (aloud and silent) in the NH and CI groups. This graph illustrates the superiority of vocal production (aloud words) over the nonproduction condition (heard words), for the NH group (aloud words: M = 0.3, SD = 0.12; silent words: M = 0.26, SD = 0.14), as well as for the CI group (aloud words: M = 0.25, SD = 0.09; silent words: M = 0.05, SD = 0.05). Note that the difference between aloud and silent words was much larger in the CI group, due to near floor-effect for the silent words.

Figure 2 Percentage of recalled words in the “auditory” learning conditions, calculated separately for the subsets of silent and aloud words, for the auditory-CI-ON and NH groups. The error bars are standard errors of their respective means.

A repeated measures ANOVA with learning condition (aloud words, silent words) as a within-subjects variable, and group (CI, NH) as a between-subjects variable, revealed a significant main effect for learning condition [F _(1,24) = 11.88, p < 0.005, η² = 0.33], and group [F _(1,24) = 26.82, p < 0.001, η² = 0.52]. A learning condition × group interaction was also found [F _(1,24) = 4.97, p < 0.05, η² = 0.17]. A post hoc analysis confirmed no PE for the NH group (p > 0.05), and a significant PE for the CI group [t ₍₁₁₎ = 6.8, p < 0.001].

Finally, comparing the three learning conditions within the CI group (auditory-CI-ON, visual-CI-ON, visual-CI-OFF) revealed a PE (advantage for the produced words) for each condition. Interestingly, the number of aloud words recalled was similar (and relatively high) across all learning conditions. However, the number of the nonproduced words (either silently read or heard) was different: results were lowest with the auditory presentation, and highest in the visual-CI-ON condition.

Statistical analysis using a repeated measures ANOVA confirmed this pattern, and showed no differences between the three learning conditions for the number of aloud words [F _(2,22) = 1.14, p > 0.05]. A significant difference was found between the learning conditions for the silent words [F _(2,22) = 14.33, p < 0.001]. For the silent words, fewer words were retrieved in the auditory condition (auditory-CI-ON) compared to the visual-CI-ON condition [t ₍₁₁₎ = 6.05, p < 0.001] and to the visual-CI-OFF condition [t ₍₁₁₎ = 3.5, p < 0.005]. No significant difference was found between the visual-CI-ON and visual-CI-OFF conditions (p > 0.05).

DISCUSSION

The study of cognitive processes (e.g., learning and memory) among young adults who received CIs in their early childhood is highly important to better understand the significant individual differences seen in a wide range of speech and language outcome measures that characterize this unique population. Knowledge about cognitive processes in CI users is necessary to identify the process of “how” they are able to use the initial sensory input conveyed by their CI, and “what” they are able to do with this information ([Pisoni and Cleary, 2003]). Clinically, such knowledge may provide new insights into rehabilitation, intervention, and educational programs.

The current findings can be summarized as follows: with the visual modality, CI users and NH peers showed similar recall rates. The PE size was also comparable. Interestingly, the implant status (ON, OFF) did not affect memory performance. Yet, with the auditory study modality, the CI group showed inferior recall rates for nonproduced (heard) words relative to the NH group. Memory performance for the produced (aloud) words was similar for both participant groups. Consequently, the PE was larger for the CI listeners relative to the NH group.

Visual Memory

Our first and second questions were whether vocal production improves memory for “visually” presented words in adults with CIs, and what is the relative impact of auditory feedback on visual memory. Based on previous findings, we expected a similar memory advantage for both groups of CI users (regardless of implant status) and for NH participants. Indeed, no differences were found between the three tested conditions (NH, visual-CI-ON, visual-CI-OFF). A superiority of vocal production over silent reading was obtained in both groups (NH, CI), and in both implant conditions (ON, OFF). These results suggest that CI users function similarly to their NH peers in the visual memory task.

This superiority of vocal production over nonproduction in memory tasks was previously reported in the great bulk of PE studies ([Forrin et al, 2012]; [Ozubko et al, 2012]) among NH individuals. The advantage of vocal production is attributed to the higher distinctiveness of the produced words relative to the nonproduced words (encoding distinctiveness account; [MacLeod et al, 2010]). Vocalization involves a larger number of distinct encoding processes (visual, motor, auditory) than silent reading does. Hence, memory for aloud words is enhanced. The present study’s results show a similar pattern among CI users, regardless of the implant ON/OFF status, confirming our hypothesis. According to our findings, CI users of this background perform in a comparable manner to NH peers when using vocalization as a learning strategy on a visual memory task, and they do not need moment-to-moment auditory feedback to gain the PE advantage; hearing oneself is not a necessary factor in the visual memory task for CI users.

These findings are consistent with the hypothesis that memory differences between NH and CI groups do not occur in the visual modality (for similar results in a reading span task of CI adolescents, see [Geers et al, 2013]). Yet, a recent study by [AuBuchon et al (2015)] revealed somewhat more complex picture. The authors compared visual forward and backward digit span tasks in NH and CI users. They found that the CI group scored significantly below the NH group in the forward digit span tasks. Such tasks reflect coding strategies related to phonological processing and rehearsal mechanisms used to maintain verbal information in short-term memory for brief periods ([Pisoni et al, 2008]); it seems these “immediate” memory mechanisms may be impaired in the CI group. However, no differences were observed between the CI and NH groups in the backward digit span tasks. The backward digit span, which are relatively difficult, reflect the contribution of controlled attention and the operations of higher-level “executive” processes that are used to transform and manipulate verbal information for later processing operations ([Pisoni et al, 2008]). Possibly, these high-order abilities are not decreased in CI users. The present PE paradigm reflects long-term memory and involves attentional processes, similar to the backward digit span tasks studied by Pisoni et al. As these abilities seem intact in CI users, this may explain why their performance was similar to that of their NH peers in our study.

Auditory Memory

The pattern of results of the auditory presentation was different. Overall memory performance was lower for the CI group relative to the NH, mainly since the former hardly recalled any silent words (floor-effect). An advantage of vocal production (aloud words) over nonproduction (heard words) was found only for the CIs. However, for the NH group, the PE was limited in size (marginally significant).

The lower memory performance of CI users may be attributable to the limitations of their auditory systems. CI listeners have less total listening experience and limited access to the full range of speech sounds. Even with amplification, their access to acoustic information is degraded. In fact, even long-term CI users (with more than 10 yr of implant use) have difficulties with speech perception in specific acoustic environments, such as listening in background noise ([Davidson et al, 2011]). As a result, the phonological representations of the speech sounds in immediate memory are weakened and underspecified. Since the capacity and efficiency of immediate memory is lessened, long-term memory functions (and other language processing operations) may be reduced as well ([Pisoni et al, 2011]). [Geers et al (2013)] concluded that early severe-to-profound auditory deprivation impairs normal development of working memory processes (which rely on verbal rehearsal and serial scanning of phonological information in short-term memory). Early implantation (partially restoring auditory sensation) is not sufficient to enable adequate development, thus difficulties are evident in adolescence as well. In light of these difficulties, the overall weak performance of the CI users in the auditory PE task is not surprising.

With the auditory PE paradigm, the study words were aurally presented. The nonproduced words were merely heard by the participants. Given the (limited) hearing abilities of the CI group, these words were probably weakly encoded, hence poorly recalled. In contradistinction, the aloud words were vocally produced by the participants. Vocal production provides ample somatosensory information (proprioceptive, tactile, and efferent feedbacks; [Postma, 2000]), presumably used to create a deeper memory trace of the produced words, thereby improving their recall rates and enabling similar performance as their NH peers. Taken together, overall memory performance (produced and nonproduced words) for the CIs was low in the auditory modality, but this group benefitted most from vocal production.

The picture was different for the NH group. First, overall memory (aloud and silent words) was higher relative to that of the CI group. Second, the PE was smaller (marginally significant). Note that a recent study that used similar auditory presentation found an advantage for vocalized words over nonproduced (heard) words in an NH group, yet its size was also relatively small ([Mama and Icht, 2016a], Expt 1). Possibly, since the dominant mode of processing speech in NH individuals is the auditory modality ([Pisoni, 2000]), the relative gain from vocal production is rather small. The lack of significant PE in the present study may also be related to the limited sample size.

Vocal Production of Aurally Presented Words as a Means to Assist Recall in CI Users

Comparing the three learning conditions of the CI group (auditory, visual-implant-ON, visual-implant-OFF) revealed an interesting pattern. A PE was obtained for each condition, namely advantage of produced (aloud) over nonproduced words. The number of aloud words was comparable across conditions. Yet, the number of the nonproduced words (silently read, heard) was higher in the visual conditions than in the auditory condition. What might be the source of the strikingly poor recall rates for silent words with the auditory presentation? We suggest that it may be related to “listening effort.”

According to the classic “working memory” model ([Baddeley and Hitch, 1998]), shared and limited cognitive resources can be distributed across various tasks, as necessary. An effortful task requires a large portion of relevant resources. Listening effort can be defined as the proportion of (limited) cognitive resources involved in interpreting the incoming acoustic signals. For example, the presence of background noise or distortions in speech signals increases cognitive demand (e.g., attention), and as a result, listening effort increases ([Schneider and Pichora-Fuller, 2000]). For CI users, interpreting the distorted auditory signal requires considerable cognitive resources, making listening effortful ([Pals et al, 2013]). Indeed, several studies showed increased listening effort for hearing-impaired individuals using hearing aids ([Desjardins and Doherty, 2013]) or CIs ([Hughes and Galvin, 2013]) relative to NH controls.

In the current study, with auditory presentation of the study list, the no-production condition involved “merely” listening, but may have been a demanding task for CI listeners. We suggest that the effortful listening required for encoding the study words reduces available cognitive resources. Since the CI users allocate a greater share of the cognitive resources to recovering the auditory input signal via the implant, fewer resources are left for encoding its linguistic content, thus memory for the nonproduced items may very well be negatively impacted.

However, encoding of the produced (aloud) words is different. Saying a word aloud involves additional encoding processes relative to listening (e.g., motor, tactile). Indeed, listening effort is high (similar to listening to the nonproduced words), but we suggest that the additional encoding processes successfully recruit the remaining cognitive resources, assisting aloud words recall, despite the effort. As a result, vocal production of the study words significantly improved their retrievability (fivefold). In fact, vocalization was so beneficial for the CI users, that it enabled them comparable memory performance to that of their NH peers.

Interestingly, all CI participants showed relatively good open-set consonant-vowel-consonant word recognition scores (>50%), yet their memory for nonproduced aurally presented words was poor. This finding supports the independence of traditional audiological criteria (outcome measures) and cognitive (memory) factors among CI users. In clinical evaluation of CI listeners, it is important to consider audiologically based outcome measures as well as the cognitive processes (attention, learning, and memory) that mediate speech perception and production ([Pisoni, 2000]).

Summary and Clinical Implications

Our findings suggest that the significant difference observed between CI and NH populations in word memory tasks using the PE paradigm is modality specific, and particular to the auditory modality only. Visual learning among CI users enables better memory performance (higher recall rates) relative to auditory learning, mainly for nonproduced words. Finally, auditory feedback does not significantly affect visual word memory in CI users.

These findings provide important implications on learning processes in hearing-impaired children and adolescents with CIs in mainstream educational settings. Since these schools adhere to an auditory–oral philosophy, most of the learning is based on the auditory modality. The current results stress the need for special educational accommodations for this population. For example, it is recommended that a CI user will vocally produce the to-be-remembered portion of study material, regardless of the learning modality used (visual, such as reading from a textbook; auditory, such as listening to a lecture). Intervention techniques that involve vocalization may be useful to accelerate perceptual learning and language development (e.g., in vocabulary instruction) among CI users.

Further investigations are needed to evaluate long-term memory abilities of CI users, in the visual as well as auditory modalities, and the effect of auditory feedback on learning and memory, which carry clinical (rehabilitation) and educational importance. In addition, future studies should evaluate memory for complex materials (e.g., text) to ensure that vocal production is an appropriate learning strategy for such tasks that are commonly encountered in educational settings.

Abbreviations

ANOVA: analysis of variance

CI: cochlear implant

M: mean

NH: normal hearing

PE: production effect

SD: standard deviation

No conflict of interest has been declared by the author(s).

Acknowledgments

The authors thank Dvora Fridman, Noga Gadish, Oriya Zax, and Tehila Shchar for collecting data for this study.

This paper was orally presented at The American Cochlear Implant Alliance (CI2015 DC) in Washington, DC, October 15–17, 2015.

• REFERENCES

• AuBuchon AM, Pisoni DB, Kronenberger WG. 2015; Short-term and working memory impairments in early-implanted, long-term cochlear implant users are independent of audibility and speech production. Ear Hear 36 (06) 733-737
• Baddeley A, Hitch G. 1998; Recent developments in working memory. Curr Opin Neurobiol 8 (02) 234-238
• Bavelier D, Dye MWG, Hauser PC. 2006; Do deaf individuals see better?. Trends Cogn Sci 10 (11) 512-518
• Boothroyd A. 1968; Statistical theory of the speech discrimination score. J Acoust Soc Am 43 (02) 362-367
• Castel AD, Rhodes MG, Friedman MC. 2013; Predicting memory benefits in the production effect: The use and misuse of self-generated distinctive cues when making judgments of learning. Mem Cognit 41 (01) 28-35
• Cleary M, Pisoni DB, Geers AE. 2001; Some measures of verbal and spatial working memory in eight- and nine-year-old hearing-impaired children with cochlear implants. Ear Hear 22 (05) 395-411
• Conway CM, Pisoni DB, Kronenberger WG. 2009; The importance of sound for cognitive sequencing abilities: the auditory scaffolding hypothesis. Curr Dir Psychol Sci 18 (05) 275-279
• Daneman M, Carpenter PA. 1980; Individual differences in working memory and reading. J Verbal Learn Verbal Behav 19 (04) 450-466
• Davidson LS, Geers AE, Blamey PJ, Tobey EA, Brenner CA. 2011; Factors contributing to speech perception scores in long-term pediatric cochlear implant users. Ear Hear 32 (01) (Suppl) 19S-26S
• Dawson PW, McKay CM, Busby PA, Clark GM. 2002; Rate-of-processing ability in children using cochlear implants and its relevance to speech perception. Cochlear Implants Int 3 (02) 126-138
• De Raeve L, Vermeulen A, Snik A. 2015; Verbal cognition in deaf children using cochlear implants: effect of unilateral and bilateral stimulation. Audiol Neurootol 20 (04) 261-266
• Desjardins JL, Doherty KA. 2013; Age-related changes in listening effort for various types of masker noises. Ear Hear 34 (03) 261-272
• Engel-Yeger B, Durr DH, Josman N. 2011; Comparison of memory and meta-memory abilities of children with cochlear implant and normal hearing peers. Disabil Rehabil 33 (09) 770-777
• Fawcett JM, Quinlan CK, Taylor TL. 2012; Interplay of the production and picture superiority effects: a signal detection analysis. Memory 20 (07) 655-666
• Figueras B, Edwards L, Langdon D. 2008; Executive function and language in deaf children. J Deaf Stud Deaf Educ 13 (03) 362-377
• Forrin ND, Macleod CM, Ozubko JD. 2012; Widening the boundaries of the production effect. Mem Cognit 40 (07) 1046-1055
• Fryauf-Bertschy H, Tyler RS, Kelsay DM, Gantz BJ, Woodworth GG. 1997; Cochlear implant use by prelingually deafened children: the influences of age at implant and length of device use. J Speech Lang Hear Res 40 (01) 183-199
• Geers AE, Nicholas J, Tye-Murray N, Uchanski R, Brenner C, Davidson LS, Toretta G, Tobey EA. 2000; Effects of communication mode on skills of long-term cochlear implant users. Ann Otol Rhinol Laryngol Suppl 185: 89-92
• Geers AE, Pisoni DB, Brenner C. 2013; Complex working memory span in cochlear implanted and normal hearing teenagers. Otol Neurotol 34 (03) 396-401
• Geers AE, Sedey AL. 2011; Language and verbal reasoning skills in adolescents with 10 or more years of cochlear implant experience. Ear Hear 32 (01) (Suppl) 39S-48S
• Hughes KC, Galvin KL. 2013; Measuring listening effort expended by adolescents and young adults with unilateral or bilateral cochlear implants or normal hearing. Cochlear Implants Int 14 (03) 121-129
• Icht M, Mama Y. 2015; The production effect in memory: a prominent mnemonic in children. J Child Lang 42 (05) 1102-1124
• Icht M, Mama Y, Algom D. 2014; The production effect in memory: multiple species of distinctiveness. Front Psychol 5: 886
• Kirk KI, Pisoni DB, Miyamoto RT. 2000. Lexical discrimination by children with cochlear implants: effects of age at implantation and communication mode. In Proceedings of the Vth International Cochlear Implant Conference . New York, NY: Thieme Medical Publishers;
• Kronenberger WG, Pisoni DB, Henning SC, Colson BG, Hazzard LM. 2011; Working memory training for children with cochlear implants: a pilot study. J Speech Lang Hear Res 54 (04) 1182-1196
• Lin OYH, MacLeod CM. 2012; Aging and the production effect: a test of the distinctiveness account. Can J Exp Psychol 66 (03) 212-216
• MacLeod CM. 2011; I said, you said: The production effect gets personal. Psychon Bull Rev 18 (06) 1197-1202
• MacLeod CM, Gopie N, Hourihan KL, Neary KR, Ozubko JD. 2010; The production effect: Delineation of a phenomenon. J Exp Psychol Learn Mem Cogn 36 (03) 671-685
• Mama Y, Fostick L, Icht M. (submitted) The impact of different background noises on the Production Effect: evidence for costs and benefits in free recall.
• Mama Y, Icht M. 2016; a Auditioning the distinctiveness account: expanding the production effect to the auditory modality reveals the superiority of writing over vocalising. Memory 24 (01) 98-113
• Mama Y, Icht M. 2016; b Influence of retrieval mode on effects of production: evidence for costs in free recall. Can J Exp Psychol 70 (02) 177-185
• Moberly AC, Lowenstein JH, Nittrouer S. 2016; Word recognition variability with cochlear implants: “perceptual attention” versus “auditory sensitivity”. Ear Hear 37 (01) 14-26
• Niparko JK, Tobey EA, Thal DJ, Eisenberg LS, Wang NY, Quittner AL, Fink NE. 2010; Spoken language development in children following cochlear implantation. JAMA 303 (15) 1498-1506
• Ozubko JD, Hourihan KL, MacLeod CM. 2012; Production benefits learning: the production effect endures and improves memory for text. Memory 20 (07) 717-727
• Pals C, Sarampalis A, Başkent D. 2013; Listening effort with cochlear implant simulations. J Speech Lang Hear Res 56 (04) 1075-1084
• Pisoni DB. 2000; Cognitive factors and cochlear implants: some thoughts on perception, learning, and memory in speech perception. Ear Hear 21 (01) 70-78
• Pisoni DB, Cleary M. 2003; Measures of working memory span and verbal rehearsal speed in deaf children after cochlear implantation. Ear Hear 24 (01) (Suppl) 106S-120S
• Pisoni DB, Cleary M. 2004. Learning, memory, and cognitive processes in deaf children following cochlear implantation. In: Zeng FG, Popper AN, Fay RR. Springer Handbook of Auditory Research: Auditory Prosthesis. New York, NY: Springer; 377-426
• Pisoni DB, Conway CM, Kronenberger W, Henning S, Anaya E. 2010. Executive function, cognitive control, and sequence learning in deaf children with cochlear implants. In: Marschark M, Spencer PE. The Oxford Handbook of Deaf Studies, Language, and Education. New York, NY: Oxford University Press; 439-457
• Pisoni DB, Conway CM, Kronenberger W, Horn DL, Karpicke J, Henning S. 2008. Efficacy and effectiveness of cochlear implants in deaf children. In: Marschark M, Hauser P. Deaf Cognition: Foundations and Outcomes. New York, NY: Oxford University Press; 52-101
• Pisoni DB, Geers AE. 2000; Working memory in deaf children with cochlear implants: correlations between digit span and measures of spoken language processing. Ann Otol Rhinol Laryngol Suppl 185: 92-93
• Pisoni DB, Kronenberger WG, Roman AS, Geers AE. 2011; Measures of digit span and verbal rehearsal speed in deaf children after more than 10 years of cochlear implantation. Ear Hear 32 (01) (Suppl) 60S-74S
• Postma A. 2000; Detection of errors during speech production: a review of speech monitoring models. Cognition 77 (02) 97-132
• Putnam AL, Ozubko JD, Macleod CM, Roediger 3rd HL. 2014; The production effect in paired-associate learning: benefits for item and associative information. Mem Cognit 42 (03) 409-420
• Schneider BA, Pichora-Fuller MK. 2000. Implications of perceptual deterioration for cognitive aging research. In: Craik FIM, Salthouse TA. Handbook of Aging and Cognition. Mahwah, NJ: Erlbaum; 155-219
• Svirsky MA, Robbins AM, Kirk KI, Pisoni DB, Miyamoto RT. 2000; Language development in profoundly deaf children with cochlear implants. Psychol Sci 11 (02) 153-158
• Tait M, Nikolopoulos TP, De Raeve L, Johnson S, Datta G, Karltorp E, Ostlund E, Johansson U, van Knegsel E, Mylanus EA, Gulpen PM, Beers M, Frijns JH. 2010; Bilateral versus unilateral cochlear implantation in young children. Int J Pediatr Otorhinolaryngol 74 (02) 206-211
• Taitelbaum-Swead R, Kishon-Rabin L, Kaplan-Neeman R, Muchnik C, Kronenberg J, Hildesheimer M. 2005; Speech perception of children using Nucleus, Clarion or Med-El cochlear implants. Int J Pediatr Otorhinolaryngol 69 (12) 1675-1683
• Wilson BS. 2015; Getting a decent (but sparse) signal to the brain for users of cochlear implants. Hear Res 322: 24-38