Neural network models of pitch perception in normal and implanted ears

Abstract

Pitch is the perceptual correlate of sound frequency and is important for using speech prosody, understanding tonal languages, and music appreciation. According to pitch perception theories, pitch is encoded by the place along the cochlea that has the maximum rate of excitation and the timing of the neural activity caused by cochlear excitation.

As one of the most successful neural prosthesis, cochlear implants (CIs) have enabled most recipients to achieve good speech perception in favourable listening conditions. Perceiving a precise pitch, however, is still a challenge for many CI users.

The goal of this study is to develop computational models of normal and CI hearing to advance our understanding of the mechanisms of pitch perception in both cases and to investigate the factors that affect pitch perception in electrical hearing.

Based on the two possible neural codes for pitch perception, two models of pitch perception were developed: the place model and the integrated model. The models simulated a common psychophysical experiment usually referred to as pitch ranking − where subjects are asked to decide which of the two sequentially presented sounds has a higher pitch – by receiving two stimuli and generating two outputs, the higher-amplitude of which would indicate the higher-pitched stimulus.

Synthesised vowels with defined pitches were the sound stimuli used in this study. An artificial neural network (ANN) constituted the core of the models. Inputs to the ANN were place pitch information for the place model and both place and temporal pitch information for the integrated model. Applying the error back-propagation algorithm, the ANN was trained on a training set of pitch pairs. The performance of the pitch perception model was measured using a previously unseen test set of pitch pairs.

Place code for pitch perception was extracted from simulated auditory peripheral outputs by averaging the rate of activity in the auditory nerve (AN) over time. An acoustical and an electrical model of the auditory periphery were used to simulate the activity of the AN in normal and CI hearing, respectively. The activity of the AN was further processed through a spiking neural network (SNN) to extract the temporal code of pitch. Synaptic connections in the SNN were modified by spike-timing-dependent-plasticity (STDP) to generate pitch-related precisely-timed neural activities in the SNN output neurons. Pooled inter-spike-interval histogram (ISIH) across the SNN output neurons was found to be indicative of pitch.

Validation of both pitch perception models was performed by comparing their performance with psychophysical results. The place model was applied to investigate the impact of stimulation field spread on pitch perception in CI hearing using two commercial sound processing strategies. Simulation results showed that 1) the model could replicate the performance of normal and average-performing CI listeners and 2) providing focused stimulation fields in CI hearing can be beneficial, depending on the type of sound processing strategy.

The integrated model was used to explore the role of and interaction between place and temporal cues in performing simulated pitch ranking tasks. Simulation results associated with the integrated model revealed that 1) temporal cues for pitch perception compensated for missing place cues in listening conditions such as a telephone conversation where low-frequency content of the signal was suppressed and 2) new strategies with improved temporal information can improve pitch perception in CI hearing, provided temporal and place information are consistent.

Although drawing general conclusions about auditory perception would eventually require psychophysical experiments, computational models of auditory perception such as this work assists in focusing human testing upon factors that demonstrate the strongest impact on the auditory performance of normal and CI listeners. This would therefore lead to the more rapid development of CI sound processing strategies.