Continuous tracking of long timescale memory replay across event boundaries
Final Report Abstract
Searching our continuous memories is a task that we regularly perform. When looking for our keys, for instance, we often go back to the last time we had them and then scan our memories in a forward direction. What are the psychological principles that guide this type of memory search? And how is such memory-scanning orchestrated on a neural level? Findings from this project suggest that the structure we encounter in our everyday life in the form of naturalistic events plays a crucial role in this process. Such events segment our naturalistic experience into meaningful chunks, e.g., we intuitively identify train rides and restaurant visits as discrete units of experience. In Michelmann et al. (2023), we prompted participants to perform memory-scanning through a naturalistic movie. Combining large behavioral online studies with a computational model of memory-scanning, we demonstrate that the search through the continuous memory of the film is shaped by its event structure. Participants start their search for an answer at the beginning of an event and then scan forward until target information is found. During scanning, when a decision threshold is exceeded, participants stop and skip forward to the beginning of the next event, where they continue their search. This mechanism is formalized in a computational model of memory-search: the "steppingstones" model. In the second pillar of this project, patients undergoing electrocorticographic recordings for clinical purposes performed this task in a naturalistic interview setting, where questions were read to them at the bedside after watching the same movie. We performed a novel transformation of the neural recordings that highlights slow neural activity patterns. Based on this transformation we then segmented the neural data that was recorded as patients viewed the movies, into state patterns. State patterns were systematically related to how independent raters in a separate norming sample perceived the event structure of the movie. Specifically, approximately 1300 ms after neural state transitions, raters were most likely to indicate that they perceived a boundary between two events. During the naturalistic interview, those state patterns were reinstated, i.e., patients' neural activity du ring the interview resembled the activity that was recorded at the corresponding moments during movie viewing. The spatial distribution of this reinstatement effect spanned known memory regions in neocortex, namely, the default mode network (DMN). Crucially, as patients performed memory-scanning, the neural state patterns that pertained to the scanned segment in the movie were recapitulated; they unfurled in a forward direction. Therein, the transition from one state to another during memory scanning was characterized by distinct neural correlates in cortex and hippocampus: power decreases in the lower frequency bands were observed at event transitions in cortex, and approximately S00ms before cortical state transitions in hippocampus. Together these findings represent a unique investigation into the fast time-scale dynamics that underly memories that span long period. They buttress the importance of considering structure in naturalistic experience in the investigation of episodic memories and lay the groundwork for future research that addresses how fast time-scale mechanisms of event perception and event segmentation scaffold memory processes.
Publications
-
Moment-by-moment tracking of naturalistic learning and its underlying hippocampo-cortical interactions. Nature Communications, 12(1).
Michelmann, Sebastian; Price, Amy R.; Aubrey, Bobbi; Strauss, Camilla K.; Doyle, Werner K.; Friedman, Daniel; Dugan, Patricia C.; Devinsky, Orrin; Devore, Sasha; Flinker, Adeen; Hasson, Uri & Norman, Kenneth A.
-
Bayesian Surprise Predicts Human Event Segmentation in Story Listening. Cognitive Science, 47(10).
Kumar, Manoj; Goldstein, Ariel; Michelmann, Sebastian; Zacks, Jeffrey M.; Hasson, Uri & Norman, Kenneth A.
-
Evidence That Event Boundaries Are Access Points for Memory Retrieval. Psychological Science, 34(3), 326-344.
Michelmann, Sebastian; Hasson, Uri & Norman, Kenneth A.
-
Test-retest reliability of the human connectome: An OPM-MEG study. Imaging Neuroscience, 1, 1-20.
Rier, Lukas; Michelmann, Sebastian; Ritz, Harrison; Shah, Vishal; Hill, Ryan M.; Osborne, James; Doyle, Cody; Holmes, Niall; Bowtell, Richard; Brookes, Matthew J.; Norman, Kenneth A.; Hasson, Uri; Cohen, Jonathan D. & Boto, Elena
-
Large language models can segment narrative events similarly to humans. Behavior Research Methods, 57(1).
Michelmann, Sebastian; Kumar, Manoj; Norman, Kenneth A. & Toneva, Mariya
