Deep Animal Linguistic Analysis (DALA) – Decoding animal communication using a hybrid approach between bioacoustics and machine learning

• Applicants: Professor Dr. Heribert Hofer; Professor Dr.-Ing. Andreas Maier
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2020 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 441257918

Today’s understanding of animal communication is very limited. We are still far away from identifying detailed patterns of animal sounds and their possible meanings. In order to determine statistically significant observations, it is necessary to analyze large amounts of animal acoustic and behavioral data. To generate sufficient observation data, audiovisual media are increasingly used to study communication and behavior of various kinds of animals. Due to the observer’s paradox, recording has to be done in an unobtrusiveway to create as little distraction as possible for the animals. As a result, such datasets contain large numbers of irrelevant signals, e.g. environmental noise, with only a small amount of animal interactions.Robust automated machine approaches enable the analysis of such large data collections. A main obstacle are the technological limitations of today’s analysis software that is available to bioacousticians, hindering rapid progress in animal communication research. Currently available bioacoustic analysis tools provide only basic features, like the display of waveforms, spectrograms, simple audio processing options, and some annotation functions.Deep Animal Linguistic Analysis (DALA) forms a bridge between the bioacoustics and machine learning research field in order to develop a new generation of open source analysis tool, capable of automatically handling large amounts of complex animal vocalizations. DALA uses recent deep learning and other pattern recognition techniques in order to be able to (1) segment and separate animal vocalizations within large and noisy datasets, (2) automatically identify an inventory of meaningful and different vocalization categories, (3) generate combinatorial linguistic (semantic and syntactic) patterns, and (4) build an animal specific language model. In this way, we will facilitate a systematic cross-comparison of the derived language model against associated situational video recordings and behaviour descriptions to isolate reappearing matches, which again allow to detect information about potentially meaningful communication and behavioral correlations. By filling this technological and methodological pattern recognition gap, the resulting methods/tools have a realistic potential to accelerate the biological/bioacoustic research field to a new level. Over the entire project, we will be in close collaboration with the Institute of Cognitive Sciences, Osnabrück and Leibnitz Institute for Zoo and Wildlife Research. The project adheres to the open science principle (open data, open source, and open access).

• DFG Programme: Research Grants