TRR 31: The Active Auditory System
Medicine
Final Report Abstract
In the real world, the auditory system constantly has to cope with a mixture of sounds from many acoustic sources. Furthermore, the output from these different sound sources in a complex acoustic scene dynamically varies over time as do the features that characterize a specific sound signal. After a sound wave has reached the receiver’s ears, it no longer is physically available for additional analysis. Thus, the auditory system has to integrate sounds from the same source over time requiring a memory for what has been presented and to segregate the sounds from the different sources that are simultaneously active. This must be achieved even if the sources provide signals dynamically changing over time. Despite all these difficulties, the auditory system appears to solve this complex task with ease. How this is done was the central research question of the CRC/TR 31 “The Active Auditory System” studying humans and animal models. The approach taken to understand the processing of sounds in auditory scene analysis involved a range of methods combining results obtained in neurophysiological and imaging studies and in the psychophysical investigation of perception, and comparing these with predictions of models simulating the perceptual mechanisms. Based on this comparison, it will be possible to conclude whether the processes observed in the experimental evaluation of the mechanisms underlying auditory scene analysis are sufficient to explain the brain’s ability to parse the signals from the different sources in natural acoustic scenes. Simulating the mechanisms underlying auditory scene analysis can pave the way to develop better technical hearing devices or human-computer interfaces. The research focused on a set of questions that were pursued in depth: (1) How does active listening support the selection (i.e., weighting) of features of sounds from a specific source that are represented by the patterns of neuronal activity in the auditory pathway. (2) How do binaural processing mechanisms support the segregation of static and moving sources? (3) How are acoustic features dynamically integrated over time? (4) How does combined audio-visual processing affect the source analysis in complex acoustic scenes? (5) How does the bottom-up feature analysis interact with top-down processes, or more specifically, how do the hypotheses generated in the bottom-up analysis and the state of the brain affect perception? (6) How do these mechanisms affect speech perception in normal hearing and hearing-impaired human subjects? The CRC has made significant progress regarding these research questions. The work of the CRC has identified the relevance of various sound features in different listening situations. Many monaural sound features, e.g., common onset of frequency components and harmonic relations between them, interact in perception determining in the binding of sound components from one source. Binaural feature analysis, e.g., interaural time and intensity differences and the coherence of the signals reaching the two ears determines the ability to segregate sources. The processes underlying feature analysis and integration simultaneously operate on multiple time scales. Time scales in the millisecond range are relevant in binaural unmasking or separation of signals based on spectro-temporal fluctuations of signal levels. Time scales in the range of seconds affect the sequential evaluation in a stream of signals allowing to segregate signals from different sources. Analysis at intermediate time scales affects audio-visual integration. Processes operate with intermediate or large time scales are characterized by the action of top-down processes that dynamically modify the bottom-up analysis. By experimental intervention, e.g., by transcranial electric stimulation, we were able to modify this analysis. The models developed for the processing of sounds in complex acoustic scenes provided for a good prediction of the speech perception of normal and hearing-impaired human subjects. The improved prediction allows a better adjustment of hearing devices. With algorithms derived from the physiological mechanisms the function of hearing devices can be further improved. Finally, by incorporating the knowledge obtained from studying the physiological mechanisms underlying auditory scene analysis into automatic speech recognition systems, robustly functioning human computer interfaces can be constructed.
Publications
- (2006) Verfahren zur Extraktion periodischer Signalkomponenten und Vorrichtung hierzu. Germany 102004045097
Hohmann V
- (2007) Monitoring and representing complex signals. U. S. Patent No. 7,254,500
Makeig S, Anemüller J
- (2007) Verfahren zur Begrenzung des Dynamikbereichs von Audiosignalen und Schaltungsanordnung hierzu. Germany 102004044565
Hohmann V
- (2008) Auditory cortical contrast enhancing by global winner-take-all inhibitory interactions. PLoS One 3(3):e1735
Kurt S, Deutscher A, Crook JM, Ohl FW, Budinger E, Moeller CK, Schulze H
(See online at https://doi.org/10.1371/journal.pone.0001735) - (2008) Multisensory interplay reveals crossmodal influences on 'sensoryspecific' brain regions, neural responses, and judgments. Neuron 57 (1):11-23
Driver J, Noesselt T
(See online at https://doi.org/10.1016/j.neuron.2007.12.013) - (2009) Auditory streaming of amplitude-modulated sounds in the songbird forebrain. J Neurophysiol 101:3212-3225
Itatani N, Klump GM
(See online at https://doi.org/10.1152/jn.91333.2008) - (2009) Effects of signal features and environmental noise on signal detection in the great tit, Parus major. Anim Behav 78:1293-1300
Pohl NU, Slabbekoorn H, Klump GM, Langemann U
(See online at https://doi.org/10.1016/j.anbehav.2009.09.005) - (2010) Active stream segregation specifically involves the left human auditory cortex. Hear Res 265(1-2):30-37
Deike S, Scheich H, Brechmann A
(See online at https://doi.org/10.1016/j.heares.2010.03.005) - (2010) Modeling cochlear dynamics: Interrelation between cochlea mechanics and psychoacoustics. J Acoust Soc Am 128:1870-1883
Epp B, Verhey JL, Mauermann, M
(See online at https://doi.org/10.1121/1.3479755) - (2010). Revision, extension, and evaluation of a binaural speech intelligibility model. The Journal of the Acoustical Society of America, 127(4), 2479-2497
Beutelmann R, Brand T, Kollmeier B
(See online at https://doi.org/10.1121/1.3295575) - (2010). Sound-induced enhancement of low-intensity vision: multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. J Neurosci 30:13609-23
Noesselt T, Tyll S, Boehler CN, Budinger E, Heinze HJ, Driver J
(See online at https://doi.org/10.1523/jneurosci.4524-09.2010) - (2011) Auditory model based direction estimation of concurrent speakers from binaural signals. Speech Communication (53), 592-605
Dietz M, Ewert SD, Hohmann V
(See online at https://doi.org/10.1016/j.specom.2010.05.006) - (2012) Spectro-temporal weighting of loudness PLoS ONE 7(11): e50184
Oberfeld D, Heeren W, Rennies J, Verhey JL
(See online at https://doi.org/10.1371/journal.pone.0050184) - (2013) Electrophysiological correlates of auditory change detection and change deafness in complex auditory scenes. Neuroimage 75:155-164
Puschmann S, Sandmann P, Ahrens J, Thorne J, Weerda R, Klump GM, Debener S, Thiel CM
(See online at https://doi.org/10.1016/j.neuroimage.2013.02.037) - (2013) Neural basis of multisensory looming signal. Neuroimage 65: 13-22
Tyll S, Bonath B, Schoenfeld MA, Heinze HJ, Ohl FW, Noesselt T
(See online at https://doi.org/10.1016/j.neuroimage.2012.09.056) - (2013) Task-related activation of the auditory cortex. in: Springer Handbook of Auditory Research, Volume: 'Neural Correlates of Auditory Cognition', Cohen, Yale E.; Popper, Arthur N.; Fay, Richard R. (Eds.):45-81
Scheich H, Brosch, M
(See online at https://doi.org/10.1007/978-1-4614-2350-8_3) - (2014) Discriminative Learning of Receptive Fields from Responses to non-Gaussian Stimulus Ensembles. PLoS ONE, 9(4), e93062
Meyer AF, Diepenbrock J-P, Happel MFK, Ohl FW, Anemüller J
(See online at https://doi.org/10.1371/journal.pone.0093062) - (2014) Dopamine-modulated recurrent corticoefferent feedback in primary sensory cortex promotes detection of behaviorally relevant stimuli. J Neurosci 34: 1234-1247
Happel MF, Deliano M, Handschuh J, Ohl FW
(See online at https://doi.org/10.1523/jneurosci.1990-13.2014) - (2014) Entrainment of Brain Oscillations by Transcranial Alternating Current Stimulation. Current Biology 24(3):333-339
Helfrich RF, Schneider TR, Rach S, Trautmann-Lengsfeld SA, Engel AK, Herrmann CS
(See online at https://doi.org/10.1016/j.cub.2013.12.041) - (2014) Evaluating auditory stream segregation of SAM tone sequences by subjective and objective psychoacoustical tasks, and brain activity. Front Neurosci 8: 119
Dolležal L-V, Brechmann A, Klump GM, Deike S
(See online at https://doi.org/10.3389/fnins.2020.569517) - (2014) Look now and hear what’s coming: on the functional role of cross-modal phase reset. Hearing Research 307:144-152
Thorne JD, Debener S
(See online at https://doi.org/10.1016/j.heares.2013.07.002) - (2014) Neural correlates of auditory streaming in an objective behavioral task. Proc Natl Acad Sci USA 111 (29):10738-10743
Itatani N, Klump GM
(See online at https://doi.org/10.1073/pnas.1321487111) - (2014) Selective modulation of interhemispheric functional connectivity by HD-tACS shapes perception. PLoS Biology 12(12):e1002031
Helfrich RF, Knepper H, Nolte G, Strüber D, Rach S, Herrmann CS, Schneider TR, Engel AK
(See online at https://doi.org/10.1371/journal.pbio.1002031) - (2015) Change in the coding of interaural time difference along the tonotopic axis of the chicken nucleus laminaris. Front Neural Circ 9:43
Palanca-Castán N, Köppl C
(See online at https://doi.org/10.3389/fncir.2015.00043) - (2015) Robust auditory localization using probabilistic inference and coherence-based weighting of interaural cues. The Journal of the Acoustical Society of America 138:2635-2648
Kayser H, Hohmann V, Ewert S, Kollmeier B, Anemüller J
(See online at https://doi.org/10.1121/1.4932588) - (2015) The auditory dynamic attending theory revisited: A closer look at the pitch comparison task. Brain Research 1626: 198-210
Bauer A-KR, Jaeger M, Thorne JD, Bendixen A, Debener S
(See online at https://doi.org/10.1016/j.brainres.2015.04.032) - (2015) The time window of multisensory integration: Relating reaction times and judgments of temporal order. Psychological Review 122 (2): 232–241
Diederich A, Colonius H
(See online at https://doi.org/10.1037/a0038696) - (2016) Hierarchy of prediction errors for auditory events in human temporal and frontal cortex. Proc Natl Acad Sci USA 113(24):6755-60
Dürschmid S, Edwards E, Reichert C, Dewar C, Hinrichs H, Heinze HJ, Kirsch HE, Dalal SS, Deouell LY, Knight RT
(See online at https://doi.org/10.1073/pnas.1525030113) - (2016) Monaural speech intelligibility and detection in maskers with varying amounts of spectro-temporal speech features. The Journal of the Acoustical Society of America 140(1):524-540
Schubotz W, Brand T, Kollmeier B, Ewert SD
(See online at https://doi.org/10.1121/1.4955079) - (2016) Rapid tuning shifts in human auditory cortex enhance speech intelligibility. Nature Communications 7:13654
Holdgraf C, de Heer W, Pasley B, Rieger J, Crone N, Lin J, Knight R, Theunissen F
(See online at https://doi.org/10.1038/ncomms13654) - (2016) Sentence recognition prediction for hearing-impaired listeners in stationary and fluctuation noise with FADE: Empowering the Attenuation and Distortion concept by Plomp with a quantitative processing model. Trends in Hearing 20
Kollmeier B, Schädler MR, Warzybok A, Meyer BT, Brand T
(See online at https://doi.org/10.1177/2331216516655795) - (2016) The multiple contributions of interaural differences to improved speech intelligibility in multitalker scenarios. J Acoust Soc Am 139:2589-2603
Schoenmaker E, Brand T, van de Par S
(See online at https://doi.org/10.1121/1.4948568) - (2016) Verfahren zum Betreiben eines elektroakustischen Systems und ein elektroakustisches System. U.S. Patent AKZ 10 2015 003 855.92015
Hiipakka M, Kollmeier B, Ernst S, Denk F
- 2016) Variation der Cochlea-Implantat Elektrodenzuordnung in Abhängigkeit von spektralen Eigenschaften mit unterschiedlichen Stimulationsraten”, U.S. Patent DE 10 2016 214 745.5
Jürgens T, Eichenauer A, Dietz M
- (2017) A probabilistic Poisson-based model accounts for an extensive set of absolute auditory threshold measurements. Hearing Research 353:135-161
Heil P, Matysiak M, Neubauer H
(See online at https://doi.org/10.1016/j.heares.2017.06.011) - (2017) Barn owls have ageless ears. Proc R Soc B 284: 20171584
Krumm B, Klump G, Köppl C, Langemann U
(See online at https://doi.org/10.1098/rspb.2017.1584) - (2017) Better-ear rating based on glimpsing. J Acoust Soc Am 142:1466-1481
Schoenmaker E, Sutojo S, van de Par S
(See online at https://doi.org/10.1121/1.5002684) - (2017) Deep Brain stimulation of the Nucleus Basalis of Meynert attenuates early EEG components associated with defective sensory gating in patients with Alzheimer disease – a two-case study. EJN
Dürschmid S, Reichert C, Kuhn J, Freund HJ, Hinrichs H, Heinze HJ
(See online at https://doi.org/10.1111/ejn.13749) - (2017) Exploring binaural hearing in gerbils (Meriones unguiculatus) using virtual headphones. Plos One 12 (4)
Tolnai S, Beutelmann R, Klump GM
(See online at https://doi.org/10.1371/journal.pone.0175142) - (2017) Measuring multisensory integration: from reaction times to spike counts. Scientific Reports 7(1):3023
Colonius H, Diederich A
(See online at https://doi.org/10.1038/s41598-017-03219-5) - (2017) Modeling speech localization, talker identification, and word recognition in a multi-talker setting. The Journal of the Acoustical Society of America 142(1):35-54
Josupeit A, Hohmann V
(See online at https://doi.org/10.1121/1.4990375) - (2017) Onset-duration matching of acoustic stimuli revisited: conventional arithmetic versus proposed geometric measures of accuracy and precision. Frontiers in Psychology 7
Friedrich B, Heil P
(See online at https://doi.org/10.3389/fpsyg.2016.02013) - (2017). Encoding and Decoding Models in Cognitive Electrophysiology. Frontiers in Systems Neuroscience, 11, 61
Holdgraf CR, Rieger JW, Micheli C, Martin S, Knight RT, Theunissen FE
(See online at https://doi.org/10.3389/fnsys.2017.00061) - (2018) Dopaminergic neuromodulation of high-gamma stimulus phase-locking in gerbil primary auditory cortex mediated by D1/D5-receptors. Eur J Neurosci.
Deliano M, Brunk MGK, El-Tabbal M, Zempeltzi MM, Happel MFK, Ohl FW
(See online at https://doi.org/10.1111/ejn.13898) - (2018) Predicting Speech Intelligibility with Deep Neural Networks. Computer Speech and Language 48:51-66
Spille C, Kollmeier B, Meyer BT
(See online at https://doi.org/10.1016/j.csl.2017.10.004)
