From sensory ecology to cognition - the multi-sensory perception and representation of objects in fish

• Applicant: Professor Dr. Gerhard von der Emde
• Subject Area: Sensory and Behavioural Biology
• Term: from 2013 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 240543582

Most animals use information from multiple senses to analyse features of their environment and to guide behaviour. A fundamental question in sensory biology is how multiple sensory systems operate together to increase the reliability of the overall perception. In our previous project we have shown that the weakly electric fish Gnathonemus petersii is capable of cross-modal object recognition, i.e. they can recognise objects cross-modally without prior training with the sense being tested. This suggests that after learning, a representation containing object features is stored in the brain and can subsequently be accessed by other senses. Furthermore we showed that multi-sensory object perception is influenced by a dynamic weighting of sensory inputs, i.e. fish weight input from single sensory modalities according to their reliability to minimise uncertainty. This weighting of sensory inputs can also lead to the dominance of a certain sense during specific situations or tasks (sensory capture). In this project we want to investigate the fundamental processes that underpin multi-sensory perception in vertebrates. Using G. petersii as a model, we will explore the rules and the neural substrates governing cross-modal object recognition and weighting of sensory input. We will use specially designed objects, which provide different information when sensed either through vision or the electric sense. In the behavioural part, we will investigate whether Gnathonemus integrates information in an optimal fashion, i.e. whether sensory weighting depends on the reliability of the particular sensory modality. By varying the training conditions we will examine whether dynamic weighing of multisensory input is influenced by prior experience or is an innate property of the perceptual system. In addition we will explore how sensory information is weighted if the different senses provide conflicting information at certain defined reliability levels. In the neuroanatomical part, we aim at finding the neural substrates for cross-modal object recognition and sensory weighting. First, we will inject selective tracers into visual and electro-sensitive sensory brain areas to define the ascending sensory systems and to be able to interpret unisensory and multisensory convergence. Prime candidates for relevant brain areas during object detection are parts of the Cerebellum, the Optic Tectum (midbrain) and the Pallium (forebrain). In addition to the tracing studies, we will perform experiments aimed at revealing neuronal activation processes during multi-sensory learning and cross-modal transfer. This is achieved by visualizing the production of activation markers (phosphorylated external signal related kinase, pERK) in the fish brain. We will define the uni- and multi-sensory sites which are active during multi-sensory object inspection. This might ultimately tell us, where and how sensory weighting takes place in the brain of G. petersii.

• DFG Programme: Research Grants