Neural and computational basis of dynamic material perception

• Applicant: Dr. Vivian Paulun
• Subject Area: Biological Psychology and Cognitive Neuroscience ; Human Cognitive and Systems Neuroscience ; Cognitive, Systems and Behavioural Neurobiology
• Term: from 2020 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 442692081

When we look at objects, we determine not only their identity and location, but what they are made of. Visual inference of material properties like softness or fragility is crucial to predicting and interacting with our environment. Yet, it is unclear how the brain achieves this remarkable feat. I will bring together expertise from two leading labs (Dr. Tenenbaum, and Dr. Kanwisher) with complementary competencies in computational modeling, brain imaging and my own experience in material perception to fundamentally advance our understanding of how we perceive and reason about materials in dynamic displays. The two-year research program is structured into four distinct work packages (WP). I will first use computer simulations and photography to create a benchmark dataset of videos depicting naturalistic material interactions, e.g. deforming objects, or pouring liquids (WP1). A subset of scenes will be used in a behavioral experiment investigating the perceptual representation of dynamic materials (WP 2) and in an fMRI task to uncover the underlying neural representations (WP3). To understand the computations entailed in visual inference about dynamic materials, we will develop a set of competing computational models trained on our benchmark dataset (WP4). We will test the correspondence between the perceptual, neural, and computational representations of dynamic materials using representational similarity analysis. Importantly, each WP will also advance our knowledge independently, because the same data will be used in complementary analyses, e.g. discovering dimensions of neural coding of material properties in dynamic scenes. The proposed research builds directly on my expertise in material perception, psychophysics, physics simulations, and computer graphics, and gives me the unique opportunity to acquire new skills in advanced modelling techniques, machine learning and neuroimaging in two world-renowned laboratories at MIT.

• DFG Programme: Research Fellowships
• International Connection: USA
• Hosts: Professorin Nancy Kanwisher; Professor Dr. Joshua B. Tenenbaum