Visuo-Linguistic Interaction in Dynamic Automotive Environments

• Applicant: Dr. Hendrik Kösling
• Subject Area: Security and Dependability, Operating-, Communication- and Distributed Systems
• Term: from 2004 to 2007
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 5425940

I propose to investigate the interaction between language and visual attention during the presentation of spoken messages of navigation systems in a dynamic automotive environment. Inspired by previous research, eye-tracking studies should now yield how different aspects of message presentation determine the comprehension of spoken messages in an animated, real-world driving scenario. The analysis of eye movements provides the spatio-temporal distribution of visual attention and should reflect visuo-linguistic interaction: How do drivers perceive a certain spoken navigation system message in a certain dynamic situation? Why does this lead to a specific message interpretation? Characteristic determinants such as the syntactic and semantic message structure and the "timing" of message issuing will be considered. I assume that message comprehension is largely dependent on the current, dynamic traffic situation. It thus appears indispensable to dynamically adapt the message design and presentation accordingly. This should ensure optimal, "safe" comprehension and, consequently, improve driving safety considerably. If spoken language can be (sequentially) mapped onto respective gaze trajectories in dynamic scenarios, I have the unique opportunity to computationally generate gaze trajectories that correspond to new messages in order to evaluate their comprehensibility. In "reverse", this model could also generate optimal spoken directives in analogy to efficient gaze trajectories. Generation and testing of enhanced message corpora would thus be greatly simplified.

• DFG Programme: Research Fellowships