Influence of spoken text on cognitive processing of complex pictorial presentations

• Applicant: Dr. Manuela Glaser
• Subject Area: Developmental and Educational Psychology
• Term: from 2017 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 343210587

Based on models of multimedia learning (Glaser & Schwan, 2015; Mayer, 2014) and psychological findings on picture and scene perception (Huettig & Altmann, 2005), the planned project examines the influence of accompanying audio texts on the perception and processing of complex pictorial contents and related knowledge acquisition processes. In four laboratory experiments it will be analyzed to which extent audio texts (1) systematically reduce the interpretational openness of pictures, (2) foster the perception and processing of neglected pictorial elements, (3) perceptually and cognitively relate pictorial elements, and (4) contribute to the dual coding of the pictorial contents. Research objects are complex and detailed history paintings about historical events, which are typically used to communicate historical knowledge (Popp, 2013). In the four studies, the influence of auditive intro texts, named salient pictorial elements, named relations between pictorial elements as well as the influence of text structure on the allocation of attention (gaze coherence, fixation times, amount of transitions, measured by eye tracking) and knowledge acquisition (free visual recall, text-picture-relation, transfer, memory related interindividual consistency) will be examined.

• DFG Programme: Research Grants
• Co-Investigator: Professor Dr. Stephan Schwan