Probabilistic inference in the primary visual cortex
Final Report Abstract
Visual perception is a strikingly difficult problem, yet our brains solve it effortlessly. The goal of this project was to investigate the mechanisms of perceptual inference in primary visual cortex (V1). We investigated how the visual stimulus (bottom-up processing) and internal states or expectations of the subject (top-down modulation) affect neural population activity. We addressed this overarching question using complementary approaches. First, we asked how well experimental paradigms that seek to quantify the effect of top-down (e.g. attentional) modulation on neuronal population activity are able to dissociate changes in attentional strength from changes in attentional state variability. We found that fluctuations in attention induce correlated neuronal variability at long timescales while attention on average reduces correlations at short timescales. These effects predominated in layers 2/3, as expected from a feedback signal such as attention. We conclude that a significant portion of correlated variability can be attributed to trial-to-trial fluctuations in internally generated signals, like attention, rather than noise. Second, building upon recent advances in deep learning, we developed a new generation of predictive models of neural activity in primary visual cortex. We found that models based on convolutional neural networks enable state-of-the-art prediction of neural activity in primary visual cortex in response to natural images. By augmenting such model architectures with interpretable components like divisive normalization, we gained insights into the role of normalization in non-linear processing within the classical receptive field of V1 neurons. Moreover, we developed an approach to cluster neurons into functional cell types and obtained preliminary evidence that V1 neurons are organized into a number of functional cell types. Third, we developed computational models of example-based visual search, a challenging computer vision problem that could also serve as a promising paradigm for future studies combining behavioral and physiological measurements in awake animals. We developed a synthetic examplebased visual search task and showed that image clutter presents a serious problem for state-of-theart computer vision systems. We extended our approach to real-world scences and investigated under which conditions trained models are able to generalize to novel object classes not seen during training. We found that a wide diversity of object classes during training is key to generalization and achieved state-of-the-art performance in one-shot visual search. In summary, the project was highly successful at developing machine learning methods for modeling neural responses in visual cortex. Combined with the strong computer vision models we developed, our results set the stage for future studies of the neural mechanisms underlying complex real-world visual inference tasks.
Publications
- (2018): Attentional fluctuations induce shared variability in macaque primary visual cortex. Nature Communications
Denfield GH, Ecker AS, Shinn TJ, Bethge M, Tolias AS
(See online at https://doi.org/10.1038/s41467-018-05123-6) - (2018): Diverse feature visualizations reveal invariances in early layers of deep neural networks. European Conference on Computer Vision (ECCV)
Cadena SA, Weis MA, Gatys LA, Bethge M, Ecker AS
(See online at https://doi.org/10.1007/978-3-030-01258-8_14) - (2018): One-Shot Segmentation in Clutter. International Conference on Machine Learning (ICML)
Michaelis C, Bethge M, Ecker AS
- (2019): A rotation-equivariant convolutional neural network model of primary visual cortex. International Conference on Learning Representations (ICLR)
Ecker AS, Sinz FH, Froudarakis E, Fahey PG, Cadena SA, Walker EY, Cobos E, Reimer J, Tolias AS, Bethge M
- (2019): Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS Computational Biology
Cadena SA, Denfield GH, Walker EY, Gatys LA, Tolias AS, Bethge M, Ecker AS
(See online at https://doi.org/10.1371/journal.pcbi.1006897) - (2021): Learning divisive normalization in primary visual cortex. PLoS Computational Biology
Burg MF, Cadena SA, Denfield GH, Walker EY, Tolias AS, Bethge M, Ecker AS
(See online at https://doi.org/10.1371/journal.pcbi.1009028)
