Temporal ventriloquism effect and auditory dominance in multimodal duration perception
Final Report Abstract
If an event stimulates several modalities at the same time, the inputs from the different modalities have to be integrated into a single and coherent perceptual impression. Only few previous studies have investigated multisensory integration processes in the domain of duration perception. These studies indicate that conflicts between duration information from different modalities are typically resolved in favour of the auditory information. In this research project, we extended this existing evidence by demonstrating that – albeit to a lesser extent – visual information also may contribute to the combined temporal percept. In addition, we provided evidence that these multisensory integration effects are likely mediated by temporal ventriloquism, and are due to a transient rather than a sustained mechanism. Interestingly, perceived duration can also be affected by spatial information, which is most reliably conveyed by the visual modality. We showed that such visuo-spatial information strongly affects perceived auditory duration. Thus, sensory dominance is not hard-wired, but varies along with contextual factors. That is, when information from different modalities is present, the extent to which one sense drives the overall percept depends on the interaction between multiple perceptual domains, such as space and time. Further experimental work addressed the locus of this multimodal integration of temporal information. Specifically, the observed multimodal integration effects might reflect genuine sensory interactions between the inputs from different modalities, but alternatively, they might also emerge within decisional or memory-related processing. On the one hand, experiments in which we directly assessed the effect of multimodal conflicting duration information on perceived duration provided evidence consistent with the notion of a genuine perceptual integration. On the other hand, experiments that assessed stimulus aspects other than perceived duration (e.g., identification of visually presented letters) did not yield any evidence for an influence of sound duration on visual accuracy, even though the mere presence of sound facilitated performance. In general, we conclude that multimodal integration of conflicting duration information is a stable phenomenon that may severely affect judgments of temporal information. At the same time, according to our results, processing of nontemporal information seems unaffected by multisensory duration integration.
Publications
- (2018) Enhancement of letter identification by concurrent auditory stimuli of varying duration. Acta psychologica 190 38–52
La Rosa, Maria D. de; Bausenhart, Karin M.
(See online at https://doi.org/10.1016/j.actpsy.2018.07.001) - (2013). Multimodal integration of interval duration: Temporal ventriloquism or changes in pacemaker rate? Timing and Time Perception, 1, 189-215
De la Rosa, M. D., & Bausenhart, K. M.
(See online at https://dx.doi.org/10.1163/22134468-00002015) - (2014). Multimodal integration of time: Visual and auditory contributions to perceived duration and sensitivity. Experimental Psychology, 61, 310-322
Bausenhart, K. M., De la Rosa, M. D., & Ulrich, R.
(See online at https://dx.doi.org/10.1027/1618-3169/a000249) - (2016). Representations of temporal information in short-term memory: Are they modality-specific? Acta Psychologica, 170, 163-167
Bratzke, D., Quinn, K. R., Ulrich, R., & Bausenhart, K. M.
(See online at https://doi.org/10.1016/j.actpsy.2016.08.002) - (2018). Assessing duration discrimination: Psychophysical methods and psychometric function analysis. In A. Vatakis, F. Balci, M. Di Luca & A. Correa (Eds.), Timing and Time Perception: Procedures, Measures, and Applications (pp. 52–78). Leiden: Brill
Bausenhart, K.M., Di Luca, M., & Ulrich, R.
(See online at https://dx.doi.org/10.1163/9789004280205_004) - (2018). On the interplay of visuospatial and audiotemporal dominance: Evidence from a multimodal Kappa effect. Attention, Perception, & Psychophysics, 80, 535-552
Bausenhart, K. M., & Quinn, K. R.
(See online at https://doi.org/10.3758/s13414-017-1437-z) - (2018). Still no evidence for sustained effects of multisensory integration of duration. Multisensory Research, 31, 601 – 622
De la Rosa, M. D., & Bausenhart, K. M.
(See online at https://doi.org/10.1163/22134808-18001296)
