The neural dynamics of causal evidence accumulation in multisensory perception (MultiCEA)

• Applicant: Professor Dr. Tim Rohe
• Subject Area: Biological Psychology and Cognitive Neuroscience ; General, Cognitive and Mathematical Psychology
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 566261189

Human perception is remarkably successful at obtaining coherent, adaptive, multisensory representations from extremely complex sensory environments. For veridical representations, our brains must integrate stimuli across modalities if they originate from a common cause, but not integrate crossmodal stimuli from independent causes. For example, at a crowded party, our brains can efficiently integrate one out of many voices with the face of a speaker, and rarely misidentify who is speaking. Recent evidence from us and others suggest that the brain infers the causal structure of multisensory stimuli (i.e., common vs. independent causes) by combining causal evidence from spatiotemporal relations between stimuli with prior assumptions. To date, most research has focused on causal inference as a static ‘one-shot’ process, with brief sensory inputs and a fixed causal inference output. However, understanding multisensory perception in complex environments requires a time-resolved framework: In an ever-changing sensory landscape, the brain must dynamically exploit intersensory relations to accumulate causal evidence over time to make fast and accurate causal decisions. We propose to develop such a dynamic framework of multisensory causal inference by intertwining two time-evolving processes: First, a causal evidence accumulation process whereby the brain samples sensory information to reduce uncertainty about the underlying causal structure of the stimuli. Second, a competitive process whereby alternative (and incompatible) hypotheses about the stimuli’s causal structure are entertained during causal evidence accumulation. To resolve the competition, the brain recruits prefrontal conflict monitoring and regulation processes. Accumulation and competition continue until causal uncertainty is sufficiently reduced, and a multisensory percept has stabilised. To investigate this neurocognitive model, the project combines audiovisual psychophysical experiments with computational modelling and uni- as well as multivariate pattern analysis of EEG and fMRI data from healthy human participants in three work packages (WPs). Using an audiovisual rate paradigm, we will characterise how the brain accumulates causal evidence by integrating Bayesian causal inference with a drift-diffusion model (CI-DDM). In a psychophysical study (WP1), the CI-DDM models observers’ causal decisions and their reaction times as well as their audiovisual perceptual integration. Applied to EEG (WP2) and fMRI data (WP3), the CI-DDM model charts the neural spatiotemporal dynamics of causal evidence accumulation and causal inference throughout the cortical hierarchies. Further, we will investigate how prefrontal conflict monitoring and regulation resolve the competition between causal hypotheses. The results of the project will extent and generalise our current understanding of how humans perform causal inference for multisensory perception in dynamic, complex environments.

• DFG Programme: Research Grants