Sound recognition with limited supervision over sensor networks

• Applicant: Professor Dr.-Ing. Reinhold Häb-Umbach
• Subject Area: Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2016 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 282835863

A general problem of machine learning systems is the mismatch between training and test data resulting in a significant degradation of performance. For sound recognition in acoustic sensor networks (ASNs) the problem is aggravated because of the huge number and variability of sounds and acoustic environments, and because of the large variety of sensor locations and geometric configurations one can encounter. Therefore, existing databases for sound recognition will almost never be a perfect fit to any concrete target application in acoustic sensor networks.The main objective of this project is to devise techniques for making use of available resources for the development of high-performance acoustic event and scene classifiers for a specific target application in an ASN. Those available resources are on one hand weakly labeled data (data annotated only with the event class, but not with temporal on/offset information), which stem from a different domain than the target domain for which an application is to be developed. On the other hand we assume availability of lots of unlabeled audio recordings from the target domain. We will develop techniques to compute strong labels (event category plus on/offset times), to compute domain-invariant features, and to carry out domain adaptation. We will also consider adaptation during test to account for dynamic environments and sensor configurations. The main methodology applied will be deep generative models. We will develop neural models and appropriate objective functions to disentangle sources of variation, in particular to separate audio content related variations from environment induced variations. Furthermore, we will develop methods to detect abnormal acoustic events, as those may be of particular interest in an application.

• DFG Programme: Research Units
• Subproject of: FOR 2457: Acoustic Sensor Networks
• Co-Investigator: Professor Dr.-Ing. Rainer Martin