Temporal ventriloquism effect and auditory dominance in multimodal duration perception

• Applicant: Dr. Karin Maria Bausenhart
• Subject Area: General, Cognitive and Mathematical Psychology
• Term: from 2011 to 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 211605914

If an event stimulates several modalities at the same time, the inputs from the different modalities have to be integrated into a single and coherent perceptual impression. Only few previous studies have investigated multisensory integration processes in the domain of time perception. These studies indicate that conflicts between duration information from different modalities are typically resolved in favour of the auditory information. In the first phase of this research project, we extended this existing evidence about auditory dominance in multisensory duration integration by demonstrating that - albeit to a lesser extent - the visual information also contributes to the combined temporal percept. In addition, we provided evidence that these multisensory integration effects are likely mediated by temporal ventriloquism, and are due to a transient rather than a sustained mechanism. Interestingly, perceived duration can also be affected by spatial information, which is most reliably conveyed by the visual modality. We showed that such visuo-spatial information strongly affects perceived auditory duration. Thus, sensory dominance is not hard-wired, but varies along with contextual factors. That is, when information from different modalities is present, the extent to which one sense drives the overall percept depends on the interaction between multiple perceptual domains, such as space and time. To extend these results, we suggest a novel series of 14 experiments addressing five research objectives: (A) a further control study is needed to strengthen our conclusion that multimodal time perception is transient rather than sustained. (B) So far, multimodal duration integration was investigated exclusively with auditory and visual stimuli. In a series of experiments, these results will be generalized to multimodal interactions including the tactile modality. On the one hand, this will broaden the scope of our research, and on the other hand, hopefully provide a useful prerequisite for the series of experiments on objective (C): the notion of optimal integration. Specifically, this notion holds that the contributions from different modalities are weighted according to the reliability of their sensory input, in order to maximise the sensitivity of the perceptual system. Finally, a crucial aspect concerning the multimodal integration of time is to investigate the locus of this process. Multimodal integration effects might reflect genuine sensory interactions between the inputs from different modalities, but alternatively, they might emerge later, on a decisional stage of processing. We propose two series of experiments to address this theoretically pivotal issue by employing (D) behavioural and (E) electrophysiological measures. These experiments will hopefully provide novel and valuable insights in the principles and mechanisms underlying the multimodal integration of perceived duration.

• DFG Programme: Research Grants