Development, Reliability, and Validity of PRESTO: A New High-Variability Sentence Recognition Test

Background: There is a pressing need for new clinically feasible speech recognition tests that are theoretically motivated, sensitive to individual differences, and access the core perceptual and neurocognitive processes used in speech perception. PRESTO (Perceptually Robust English Sentence Test Open-set) is a new high-variability sentence test designed to reflect current theories of exemplar-based learning, attention, and perception, including lexical organization and automatic encoding of indexical attributes. Using sentences selected from the TIMIT (Texas Instruments/Massachusetts Institute of Technology) speech corpus, PRESTO was developed to include talker and dialect variability. The test consists of lists balanced for talker gender, keywords, frequency, and familiarity.

Purpose: To investigate the performance, reliability, and validity of PRESTO.

Research Design: In Phase I, PRESTO sentences were presented in multitalker babble at four signal-to-noise ratios (SNRs) to obtain a distribution of performance. In Phase II, participants returned and were tested on new PRESTO sentences and on HINT (Hearing In Noise Test) sentences presented in multitalker babble.

Study Sample: Young, normal-hearing adults (N = 121) were recruited from the Indiana University community for Phase I. Participants who scored within the upper and lower quartiles of performance in Phase I were asked to return for Phase II (N = 40).

Data Collection and Analysis: In both Phase I and Phase II, participants listened to sentences presented diotically through headphones while seated in enclosed carrels at the Speech Research Laboratory at Indiana University. They were instructed to type in the sentence that they heard using keyboards interfaced to a computer. Scoring for keywords was completed offline following data collection. Phase I data were analyzed by determining the distribution of performance on PRESTO at each SNR and at the average performance across all SNRs. PRESTO reliability was analyzed by a correlational analysis of participant performance at test (Phase I) and retest (Phase II). PRESTO validity was analyzed by a correlational analysis of participant performance on PRESTO and HINT sentences tested in Phase II, and by an analysis of variance of within-subject factors of sentence test and SNR, and a between-subjects factor of group, based on level of Phase I performance.

Results: A wide range of performance on PRESTO was observed; averaged across all SNRs, keyword accuracy ranged from 40.26 to 76.18% correct. PRESTO accuracy at retest (Phase II) was highly correlated with Phase I accuracy (r = 0.92, p < 0.001). PRESTO scores were also correlated with scores on HINT sentences (r = 0.52, p < 0.001). Phase II results showed an interaction between sentence test type and SNR [F(3, 114) = 121.36, p < 0.001], with better performance on HINT sentences at more favorable SNRs and better performance on PRESTO sentences at poorer SNRs.

Conclusions: PRESTO demonstrated excellent test/retest reliability. Although a moderate correlation was observed between PRESTO and HINT sentences, a different pattern of results occurred with the two types of sentences depending on the level of the competition, suggesting the use of different processing strategies. Findings from this study demonstrate the importance of high-variability materials for assessing and understanding individual differences in speech perception.

Key Words

Individual differences - speech recognition - talker variability