Multisensory Stimuli Switching and Modification for Augmented Reality in Dynamic Conditions

• Applicant: Professor Dr. Ernst Kruijff
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 503868504

Over the last decade, Augmented reality (AR) has increasingly attracted the interest of researchers, the general public and companies. AR systems supplement the real world with computer-generated objects like overlays, text labels or guidance cues. Even though stationary usage is not uncommon, mobile AR usage is widespread, where users move around to explore indoor and outdoor spaces. In particular head-worn devices support mobile and hands-free usage where users can simultaneously move around, keep track of augmentations and interact with the real world. Mobile users often need to operate under highly dynamic and uncontrolled external conditions, associated with the usage environment (e.g., different visual conditions) and varying user states (e.g., attention state, cognitive load). Task performance under these conditions can be challenging: a better understanding of perceptual and cognitive processes that affect task performance during the dynamic conditions prevalent in mobile AR usage – as well as their interplay – likely is essential to design optimal AR systems that perform well under these conditions. In this project, we aim at analysing the effect of and interplay between environmental and user state conditions and optimize augmentations accordingly to improve task performance. The key and novelty of our proposal is to optimize view management in AR by using multisensory stimuli for the various types of augmentations that can be switched or modified over time based on the conditions. The usage of multisensory techniques in dynamic conditions poses numerous novel challenges specific to AR. This project focuses at the effect of multisensory stimuli on search performance, awareness and cognitive load in AR view management during dynamic conditions.

• DFG Programme: Research Grants