Combining active vision with multimodal speech comprehension (C05)

• Subject Area: General, Cognitive and Mathematical Psychology
• Term: from 2016 to 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 261402652

Project C5 uses eye tracking to investigate how humans integrate visual clues from the environment while they simultaneously hear speech that refers to that environment. The goal is to use insights from this investigation not only to improve existing computational models of visually informed language comprehension, but also to increase the effectiveness of crossmodal learning in humans. Experiments will explore how eye movements are modulated by cognitive activity and influenced by culture.

• DFG Programme: CRC/Transregios
• Subproject of: TRR 169: Crossmodal Learning: Adaptivity, Prediction and Interaction
• International Connection: China
• Applicant Institution: Universität Hamburg
• Co-Applicant Institution: Chinese Academy of Sciences
• Project Heads: Professor Dr. Xingshan Li; Professor Dr. Wolfgang Menzel; Professorin Dr. Qingqing Qu