Neural dynamics of visual cognition
Cognitive, Systems and Behavioural Neurobiology
Final Report Abstract
In the blink of an eye we effortlessly recognize everyday objects such as the letters on this page or a coffee cup on a table. The apparent ease of our visual recognition abilities, however, belies its complexity: we correctly identify and classify objects among thousands of alternatives, rapidly within a few hundred milliseconds, and despite huge variation in the retinal image created by changes in viewing conditions. The aim of this project was to shed light on the neural dynamics that makes visual object perception possible. For this we used a combination of non-invasive human brain imaging, computational modelling (deep neural networks), behavioral assessment and multivariate analysis techniques. In particular, we aimed at revealing a) the complex spatiotemporal dynamics enabling object perception (objective 1), b) the way information flows through the underlying neural architecture in feedforward and feedback direction (objective 2), and c) how plastic the underlying architecture is (objective 3). We have obtained the following results. Concerning objective 1, we have a) shed light on the algorithms and representational format of the visual neural processing cascade by demonstrating a spatiotemporal correspondence between human visual brain activity and artificial deep neural networks, b) revealed spatiotemporal dynamics for different visual contents, dimensions and formats, c) identified the subset of activity that is suitably formatted to guide behavior in the effects made visible to the experimenter, d) showed how real-world regularities shape the neural architecture underlying object and scene vision, and e) sharpened analysis methods to make available relevant pieces of empirical information for theory. Concerning objective 2, we have revealed a) that veridical vision (i.e., feed-forward related information flow) and imagery (feedback-related information flow) share neural representations in the alpha frequency band, b) that task goals exhibited late effects on object processing by selectively enhancing task-relevant object features, with limited impact on the overall pattern of object representations, c) that feed-forward and feedback information flow-related activity can be dissociated through rapid serial visual presentation, d) and that alpha frequency feedback orchestrates coherent natural vision. Towards objective 3, we have revealed the spatiotemporal dynamics of the transformation from sensory to perceptual Braille letter perception in the blind, and we have ascertained the behavioral relevance of the representations revealed. Together, our results contribute to the understanding of how the complex spatiotemporal neural dynamics in the visual brain enable our perception of the world and our interaction with it in a meaningful way.
Publications
-
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific Reports, 6(1).
Cichy, Radoslaw Martin; Khosla, Aditya; Pantazis, Dimitrios; Torralba, Antonio & Oliva, Aude
-
Similarity-Based Fusion of MEG and fMRI Reveals Spatio-Temporal Dynamics in Human Cortex During Visual Object Recognition. Cerebral Cortex, 26(8), 3563-3579.
Cichy, Radoslaw Martin; Pantazis, Dimitrios & Oliva, Aude
-
Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks. NeuroImage, 153, 346-358.
Martin, Cichy Radoslaw; Khosla, Aditya; Pantazis, Dimitrios & Oliva, Aude
-
Multivariate pattern analysis of MEG and EEG: A comparison of representational structure in time and space. NeuroImage, 158, 441-454.
Cichy, Radoslaw Martin & Pantazis, Dimitrios
-
Resolving the neural dynamics of visual and auditory scene processing in the human brain: a methodological approach. Philosophical Transactions of the Royal Society B: Biological Sciences, 372(1714), 20160108.
Cichy, Radoslaw Martin & Teng, Santani
-
Decoding the orientation of contrast edges from MEG evoked and induced responses. NeuroImage, 180, 267-279.
Pantazis, Dimitrios; Fang, Mingtong; Qin, Sheng; Mohsenzadeh, Yalda; Li, Quanzheng & Cichy, Radoslaw Martin
-
Finding decodable information that can be read out in behaviour. NeuroImage, 179, 252-262.
Grootswagers, Tijl; Cichy, Radoslaw M. & Carlson, Thomas A.
-
Multivariate pattern analysis for MEG: A comparison of dissimilarity measures. NeuroImage, 173, 434-447.
Guggenmos, Matthias; Sterzer, Philipp & Cichy, Radoslaw Martin
-
Scale-specific analysis of fMRI data on the irregular cortical surface. NeuroImage, 181, 370-381.
Chen, Yi; Cichy, Radoslaw Martin; Stannat, Wilhelm & Haynes, John-Dylan
-
The representational dynamics of task and object processing in humans. eLife, 7.
Hebart, Martin N.; Bankson, Brett B.; Harel, Assaf; Baker, Chris I. & Cichy, Radoslaw M.
-
Tracking the Spatiotemporal Neural Dynamics of Real-world Object Size and Animacy in the Human Brain. Journal of Cognitive Neuroscience, 30(11), 1559-1576.
Khaligh-Razavi, Seyed-Mahdi; Cichy, Radoslaw Martin; Pantazis, Dimitrios & Oliva, Aude
-
Typical retinotopic locations impact the time course of object coding. NeuroImage, 176, 372-379.
Kaiser, Daniel; Moeskops, Merle M. & Cichy, Radoslaw M.
-
Typical visual-field locations enhance processing in object-selective channels of human occipital cortex. Journal of Neurophysiology, 120(2), 848-853.
Kaiser, Daniel & Cichy, Radoslaw M.
-
Typical visual-field locations facilitate access to awareness for everyday objects. Cognition, 180, 118-122.
Kaiser, Daniel & Cichy, Radoslaw M.
-
Ultra-Rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway. eLife, 7.
Mohsenzadeh, Yalda; Qin, Sheng; Cichy, Radoslaw M & Pantazis, Dimitrios
-
A neural mechanism for contextualizing fragmented inputs during naturalistic vision. eLife, 8.
Kaiser, Daniel; Turini, Jacopo & Cichy, Radoslaw M.
-
Cortical sensitivity to natural scene structure. Human Brain Mapping, 41(5), 1286-1295.
Kaiser, Daniel; Häberle, Greta & Cichy, Radoslaw M.
-
Deep Neural Networks as Scientific Models. Trends in Cognitive Sciences, 23(4), 305-317.
Cichy, Radoslaw M. & Kaiser, Daniel
-
Object Vision in a Structured World. Trends in Cognitive Sciences, 23(8), 672-685.
Kaiser, Daniel; Quek, Genevieve L.; Cichy, Radoslaw M. & Peelen, Marius V.
-
Reliability and Generalizability of Similarity-Based Fusion of MEG and fMRI Data in Human Ventral and Dorsal Visual Streams. Vision, 3(1), 8.
Mohsenzadeh, Yalda; Mullin, Caitlin; Lahner, Benjamin; Cichy, Radoslaw Martin & Oliva, Aude
-
The Algonauts Project. Nature Machine Intelligence, 1(12), 613-613.
Cichy, Radoslaw Martin; Roig, Gemma & Oliva, Aude
-
The Neural Dynamics of Familiar Face Recognition. Cerebral Cortex.
Ambrus, Géza Gergely; Kaiser, Daniel; Cichy, Radoslaw Martin & Kovács, Gyula
-
The spatiotemporal neural dynamics underlying perceived similarity for real-world objects. NeuroImage, 194, 12-24.
Cichy, Radoslaw M.; Kriegeskorte, Nikolaus; Jozwik, Kamila M.; van den Bosch, Jasper J.F. & Charest, Ian
-
A M/EEG-fMRI Fusion Primer: Resolving Human Brain Responses in Space and Time. Neuron, 107(5), 772-781.
Cichy, Radoslaw M. & Oliva, Aude
-
Duality Diagram Similarity: A Generic Framework for Initialization Selection in Task Transfer Learning. Lecture Notes in Computer Science, 497-513.
Dwivedi, Kshitij; Huang, Jiahui; Cichy, Radoslaw Martin & Roig, Gemma
-
Rapid contextualization of fragmented scene information in the human visual system. NeuroImage, 219, 117045.
Kaiser, Daniel; Inciuraite, Gabriele & Cichy, Radoslaw M.
-
Real-world structure facilitates the rapid emergence of scene category information in visual brain signals. Journal of Neurophysiology, 124(1), 145-151.
Kaiser, Daniel; Häberle, Greta & Cichy, Radoslaw M.
-
Temporal dynamics of visual representations in the infant brain. Developmental Cognitive Neuroscience, 45, 100860.
Bayet, Laurie; Zinszer, Benjamin D.; Reilly, Emily; Cataldo, Julia K.; Pruitt, Zoe; Cichy, Radoslaw M.; Nelson, Charles A. & Aslin, Richard N.
-
Unravelling Representations in Scene-selective Brain Regions Using Scene Parsing Deep Neural Networks.
Dwivedi, Kshitij; Martin, Cichy Radoslaw & Roig, Gemma
-
Visual Imagery and Perception Share Neural Representations in the Alpha Frequency Band. Current Biology, 30(13), 2621-2627.e5.
Xie, Siying; Kaiser, Daniel & Cichy, Radoslaw M.
-
Benchmarking GE-BOLD, SE-BOLD, and SS-SI-VASO sequences for depth-dependent separation of feedforward and feedback signals in high-field MRI.
Iamshchinina, Polina; Haenelt, Daniel; Trampel, Robert; Weiskopf, Nikolaus; Kaiser, Daniel & Cichy, Radoslaw M.
-
Coherent natural scene structure facilitates the extraction of task-relevant object information in visual cortex. NeuroImage, 240, 118365.
Kaiser, Daniel; Häberle, Greta & Cichy, Radoslaw M.
-
Dissociable Components of Information Encoding in Human Perception. Cerebral Cortex, 31(12), 5664-5675.
Vidaurre, Diego; Cichy, Radoslaw M & Woolrich, Mark W.
-
Parts and Wholes in Scene Processing. Journal of Cognitive Neuroscience, 34(1), 4-15.
Kaiser, Daniel & Cichy, Radoslaw M.
-
Perceived and mentally rotated contents are differentially represented in cortical depth of V1. Communications Biology, 4(1).
Iamshchinina, Polina; Kaiser, Daniel; Yakupov, Renat; Haenelt, Daniel; Sciarra, Alessandro; Mattern, Hendrik; Luesebrink, Falk; Duezel, Emrah; Speck, Oliver; Weiskopf, Nikolaus & Cichy, Radoslaw Martin
-
Representational Content of Oscillatory Brain Activity during Object Recognition: Contrasting Cortical and Deep Neural Network Hierarchies. eneuro, 8(3), ENEURO.0362-20.2021.
Reddy, Leila; Cichy, Radoslaw Martin & VanRullen, Rufin
-
Semantic scene-object consistency modulates N300/400 EEG components, but does not automatically facilitate object representations. Cerebral Cortex, 32(16), 3553-3567.
Chen, Lixiang; Cichy, Radoslaw Martin & Kaiser, Daniel
-
Temporal uncertainty enhances suppression of neural responses to predictable visual stimuli. NeuroImage, 239, 118314.
Nara, Sanjeev; Lizarazu, Mikel; Richter, Craig G.; Dima, Diana C.; Cichy, Radoslaw M.; Bourguignon, Mathieu & Molinaro, Nicola
-
The Algonauts Project 2021 Challenge: How the Human Brain Makes Sense of a World in Motion
Cichy R. M.; Dwivedi K.; Lahner B.; Lascelles A.; Iamshchinina P.; Graumann M.; Andonian A.; Murty N. A. R.; Kay K.; Roig G. & Oliva A.
-
Unveiling functions of the visual cortex using task-specific deep neural networks. PLOS Computational Biology, 17(8), e1009267.
Dwivedi, Kshitij; Bonner, Michael F.; Cichy, Radoslaw Martin & Roig, Gemma
-
A large and rich EEG dataset for modeling human visual object recognition. NeuroImage, 264, 119754.
Gifford, Alessandro T.; Dwivedi, Kshitij; Roig, Gemma & Cichy, Radoslaw M.
-
A multivariate comparison of electroencephalogram and functional magnetic resonance imaging to electrocorticogram using visual object representations in humans. Frontiers in Neuroscience, 16.
Ebrahiminia, Fatemeh; Cichy, Radoslaw Martin & Khaligh-Razavi, Seyed-Mahdi
-
Disentangling five dimensions of animacy in human brain and behaviour. Communications Biology, 5(1).
Jozwik, Kamila M.; Najarro, Elias; van den Bosch, Jasper J. F.; Charest, Ian; Cichy, Radoslaw M. & Kriegeskorte, Nikolaus
-
Modelling brain representations of abstract concepts. PLOS Computational Biology, 18(2), e1009837.
Kaiser, Daniel; Jacobs, Arthur M. & Cichy, Radoslaw M.
-
Resolving the time course of visual and auditory object categorization. Journal of Neurophysiology, 127(6), 1622-1628.
Iamshchinina, Polina; Karapetian, Agnessa; Kaiser, Daniel & Cichy, Radoslaw M.
-
The spatiotemporal neural dynamics of object location representations in the human brain. Nature Human Behaviour, 6(6), 796-811.
Graumann, Monika; Ciuffi, Caterina; Dwivedi, Kshitij; Roig, Gemma & Cichy, Radoslaw M.
-
The Spatiotemporal Neural Dynamics of Object Recognition for Natural Images and Line Drawings. The Journal of Neuroscience, 43(3), 484-500.
Singer, Johannes J.D.; Cichy, Radoslaw M. & Hebart, Martin N.
-
Time-resolved multivariate pattern analysis of infant EEG data: A practical tutorial. Developmental Cognitive Neuroscience, 54, 101094.
Ashton, Kira; Zinszer, Benjamin D.; Cichy, Radoslaw M.; Nelson, Charles A.; Aslin, Richard N. & Bayet, Laurie
-
Visual category representations in the infant brain. Current Biology, 32(24), 5422-5432.e6.
Xie, Siying; Hoehl, Stefanie; Moeskops, Merle; Kayhan, Ezgi; Kliesch, Christian; Turtleton, Bert; Köster, Moritz & Cichy, Radoslaw M.
-
Alpha-frequency feedback to early visual cortex orchestrates coherent naturalistic vision. Science Advances, 9(45).
Chen, Lixiang; Cichy, Radoslaw M. & Kaiser, Daniel
-
BOLD Moments: modeling short visual events through a video fMRI dataset and metadata.
Lahner, Benjamin; Dwivedi, Kshitij; Iamshchinina, Polina; Graumann, Monika; Lascelles, Alex; Roig, Gemma; Gifford, Alessandro Thomas; Pan, Bowen; Jin, SouYoung; Ratan, Murty N. Apurva; Kay, Kendrick; Oliva, Aude & Cichy, Radoslaw
-
Deep Neural Networks and Visuo-Semantic Models Explain Complementary Components of Human Ventral-Stream Representational Dynamics. The Journal of Neuroscience, 43(10), 1731-1741.
Jozwik, Kamila M.; Kietzmann, Tim C.; Cichy, Radoslaw M.; Kriegeskorte, Nikolaus & Mur, Marieke
-
Empirically identifying and computationally modelling the brain-behaviour relationship for human scene categorization.
Karapetian, Agnessa; Boyanova, Antoniya; Pandaram, Muthukumar; Obermayer, Klaus; Kietzmann, Tim C. & Cichy, Radoslaw M.
-
Independent spatiotemporal effects of spatial attention and background clutter on human object location representations. NeuroImage, 272, 120053.
Graumann, Monika; Wallenwein, Lara A. & Cichy, Radoslaw M.
-
Laminar dissociation of feedforward and feedback signals in high-level ventral visual cortex during imagery and perception.
Carricarte, Tony; Iamshchinina, Polina; Trampel, Robert; Chaimow, Denis; Weiskopf, Nikolaus & Cichy, Radoslaw M.
-
Net2Brain: A Toolbox to compare artificial vision models with human brain responses. Journal of Vision, 23(9), 4935.
Bersch, Domenic; Dwivedi, Kshitij; Vilas, Martina; Cichy, Radoslaw Martin & Roig, Gemma
-
The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes
Gifford, A. T.; Labner, B.; Saba-Sadiqa, S.; Vilas, M. G.; Lascelles, A.; Oliva, A.; Kay, K.; Roig, G. & Cichy, R. M.
-
WITHDRAWN: The influence of the bullseye versus standard fixation cross on eye movements and classifying natural images from EEG.
Häberle, Greta; Çelikkol, Aynur Pelin & Cichy, Radoslaw M.
-
Communicative signals during joint attention promote neural processes of infants and caregivers. Developmental Cognitive Neuroscience, 65, 101321.
Bánki, Anna; Köster, Moritz; Cichy, Radoslaw Martin & Hoehl, Stefanie
-
Individual differences in internal models explain idiosyncrasies in scene perception. Cognition, 245, 105723.
Wang, Gongting; Foxwell, Matthew J.; Cichy, Radoslaw M.; Pitcher, David & Kaiser, Daniel
-
The transformation of sensory to perceptual braille letter representations in the visually deprived brain.
Haupt, Marleen; Graumann, Monika; Teng, Santani; Kaltenbach, Carina & Cichy, Radoslaw M.
