Hooray – Exploring the Performance Boundaries of a Head Worn Microphone-Array for Deep Learning based Dynamic Acoustic Scene Analysis

• Applicant: Professor Dr. Jürgen Peissig
• Subject Area: Acoustics
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 517437545

The aim of the project is to answer the question of whether head-mounted microphone arrays with AI signal processing can surpass the current state of the art in terms of dynamic sound scene analysis to that of the beamforming technologies previously used in hearing aids or telephone headsets. Furthermore, we want to investigate what performance gain can be expected when using up to 16 microphones distributed around the head in combination with deep learning models. The generalisation to arbitrary scenarios and spaces as well as the influences of dynamic parameters such as time-variant head movements under controlled conditions with different acoustic scene complexities will be addressed. The neural network should have both the dynamically varied acoustic signals and the dynamic head position at its disposal. In addition to evaluating model performance in terms of accuracy and generalisability using classical metrics, we also aim to analyse the models using neural network interpretability (interpretable AI) techniques to gain a deeper understanding of decision making. On the one hand, we will investigate the plausibility of the deep learning models by identifying known acoustic parameters and their influence on the localisation result. On the other hand, we will try to identify new dynamical parameters that are crucial for a good localisation result and could be included in existing classical localisation models.

• DFG Programme: Research Grants