Exploring the limits of behavioral complexity in rats: a novel experimental approach via reinforcement learning and information theory
Bioinformatics and Theoretical Biology
Final Report Abstract
How does the brain enable rats to behave in complex, adaptive ways in their natural environments? Neuroscientists have been struggling to study this question for two main reasons: firstly, the technology to recreate naturalistic environments suitable for systematic studies in the laboratory has been lacking, and secondly, methods for the analysis of complex behaviors have been scarce. This research project addressed both points. To study the complex behavior and brain activity of rats in a naturalistic environment, we created the Rat Interactive Foraging Facility (RIFF), a large, circular arena with twelve nose-poking ports, where rats perform various tasks. Wireless recording technology is used to record brain activity concurrently with rat behavior. The tasks implemented in the RIFF can be very complex: for instance, the availability of food/water rewards can depend on the rat’s position as tracked by a ceiling-mounted camera, timing of motion, or sequence of nose-pokes performed. We documented rats’ capability of quickly learning complex tasks in the RIFF. Within two sessions, rats learned stereotypic patterns of movement ensuring high reward rates. We also identified aspects of learning that were predominantly responsible for performance increases from the first to the second session. We used silicon shanks to record neuronal activity in auditory cortex and insular cortex. In both regions, we found sound-responsive neurons. Somewhat surprisingly, we also found location-specific neurons in the insular cortex. These neurons could only be detected due to the high temporal and spatial resolution of the RIFF. Additionally, in both insular cortex and auditory cortex, we describe neurons that are modulated by the angle between the rat’s body and its head, as extracted from the time-synchronized rat video. Thus, we established the practical usability of the RIFF for electrophysiology in freely moving rats performing complex tasks. To create a normative model of rat behavior in a complex task, we adopted an information-theoretic approach. We describe a rat task as a Markov Decision Process (MDP). In an MDP, the environment is described by well-defined states, and an agent (here, the rat) takes well-defined actions that make the environment transition from one state to the next. Rat behavior is not deterministic. We hypothesized that rats approximately optimize a complexity-value trade-off. Therefore, we used information-limited policies optimal at their respective complexity to model rat behavior. By comparing a behavioral session of a rat to a large database of information-limited policies, we were able to estimate a complexity parameter and two behavioral parameters from each rat-session. Interestingly, the evolutions of these parameters across long time scales (up to 400 days) showed a relatively fast optimization of nose-pokes and sharp-angled body rotations, and a much more long-lasting optimization of behavioral complexity. We compared firing rates in the insular cortex of rats to estimated behavioral complexity within sessions, in 10 min time-windows. This revealed that a significant fraction of insular cortex neurons is strongly correlated with behavioral complexity at this timescale. Control analyses showed that reward rates do not fully account for these correlations. This research project encompasses several main contributions. Besides the validation of an innovative behavioral arena for rats, a main novelty is the presentation of a normative model for rat behavior that includes a quantification of the complexity-value trade-off. The main neuroscientific finding is the discovery of complexity-sensitive neurons in insular cortex.
Publications
-
An automated, low-latency environment for studying the neural basis of behavior in freely moving rats. BMC Biology, 21(1).
Jankowski, Maciej M.; Polterovich, Ana; Kazakov, Alex; Niediek, Johannes & Nelken, Israel
