Towards a mechanistic understanding of dynamic object perception via high-throughput psychophysics and artificial neural networks

• Applicant: Dr. Lynn Katrina Annika Sörensen
• Subject Area: General, Cognitive and Mathematical Psychology ; Biological Psychology and Cognitive Neuroscience
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 547591872

Humans make sense of their perpetually evolving environment with such effortlessness and (usually) accuracy that it belies the complexity of inferring a myriad of objects, agents, and their interactions on a moment-by-moment basis from a stream of sensory information. Dynamic object perception - the mechanisms by which our brains continuously construct and update such representations - enables us to perceive, plan, and act in synchrony with such a dynamic world. Yet, it is unclear how the brain achieves this remarkable feat. Guided by one of the leaders in the field of object recognition (Dr. DiCarlo), I aim to understand the mechanisms underlying human dynamic object perception. The proposed two-year research program is structured in three, increasingly ambitious aims. I will first quantitatively characterize key aspects of human object perception for a large, varied, and naturalistic set of videos using an innovative continuous report paradigm (Aim 1). To identify the underlying mechanisms, I will next evaluate state-of-the-art video-capable artificial neural network models as mechanistic hypotheses of dynamic object perception (Aim 2). By surveying such existing models for their ability to predict human dynamic object reports in videos and by improving upon existing models using mechanistic motifs from Neuroscience, I aim to deliver computational models of human dynamic object perception. Finally, to uncover the key dimensions of dynamic object perception, I will leverage the most promising computational models to perform high-throughput in-silico experiments to create videos that precisely alter the dynamics of human object category reports (Aim 3). I will verify these model-predicted phenomena in another human behavioral experiment. Collectively, the gained insights promise to not only deepen our understanding of human dynamic object perception but also catalyze advancements in related fields such as the study of neural circuits and human-machine alignment for dynamic visual content. The proposed research harnesses recent technological advances in video-capable artificial neural networks and builds directly on my expertise in developing models of human visual cognition. Implementing this research will give me the unique opportunity to acquire new skills in large-scale online psychophysics and some of the most advanced computational modeling techniques in a world-renowned laboratory at MIT.

• DFG Programme: WBP Fellowship
• International Connection: USA
• Host: Professor James J. DiCarlo, Ph.D.