Should I stay or should I go? A neural system responsible for foraging decisions
Final Report Abstract
In everyday life, we are continuously confronted with the decision of whether to stay engaged with our current behavior or switch to a new course of action. For example, should I keep my old car, whose utility continuously decreases, or should I invest money to buy a new one? This decision depends on the subjective loss of staying with the old car, the cost of a new one, and what cars are available on the market, among other variables. Despite the importance of these stay-or-leave decisions, most studies into the neuronal basis of decision making have focused on choices based on either perceptual or value information. The objective of the project was to investigate the neuronal basis of stay-or-leave decisions in a paradigm inspired by a classic foraging theory from neuroethology– the Marginal Value Theorem (MVT). Studies analyzing decision making in a foraging framework consistently report that foraging behavior can be adequately described by the decision rule proposed by the MVT. On a neuronal level, several imaging studies, as well as single unit recordings and lesion studies, point to an involvement of a part of the prefrontal cortex (anterior cingulate cortex, ACC) in foraging decisions. However, a comprehensive understanding linking theoretical, electrophysiological (correlative) and inactivation (causal) data is missing. To fill this gap, we have done extensive work with behavioral studies where we show with sophisticated manipulations that mice behavior is on average well described by the Marginal Value Theorem. However, on a trial-to-trial basis animals seem to update their estimate of the average reward rate locally and compare this value to the next expected reward. We substantiate these ideas with a deep reinforcement model that was very well accepted in the machine learning community. To investigate the neuronal algorithm underlying these decisions, I recorded the activity of parvalbumin positive interneurons in the anterior cingulate cortex and show that their activity is predictive for the animals’ stay-or-leave decision. Finally, I optogenetically manipulated the activity of these neurons during the foraging task and observed effects that are in line with what we observed in the photometry recordings, i.e. later leaving with higher PV interneuron activity.
Publications
- R-learning in actor-critic model offers a biologically relevant mechanism for sequential decision-making; Part of: Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
Sergey Shuvaev, Sarah Starosta, Duda Kvitsiani, Adam Kepecs, Alexei Koulakov
