Integrating emotional states into mental representations of the environment
Final Report Abstract
When we listen to a story, we effortlessly understand each sentence within the context of an ongoing narrative. The aims of this project were to investigate how processing the meaning of current information is influenced by contextual information of the past. We used electrocorticography (ECoG) to measure time-resolved neural activity across the lateral surface of the human brain, while the participants performed a working memory task. Briefly, participants had to listen to a pair of sentences, an initial context-setting sentence followed by a target sentence, then silently rehearse the target sentence once verbatim in their mind, and finally vocalize the target sentence aloud. Each target sentence could be preceded by one of two different context sentences, which would provide two different semantic contexts for interpreting the target sentence. We hypothesized that the responses in higher-order semantic regions like the prefrontal cortex or temporoparietal junction would be sensitive to prior contextual information, enabling us to differentiate contextual semantic differences of identical phonological input in these regions. As a control condition, we also presented target sentences that were semantically incoherent and did not relate to the context sentence. We observed greater broadband power responses (70-200 Hz) for coherent over incoherent target sentences in the middle and inferior temporal gyrus, temporoparietal cortex and in dorsal frontal areas, strongly resembling the semantic network. To assess sentence-specific temporal activation patterns, we adopted a linear decomposition technique called Correlated Component Analysis. In a cross-validation approach, we used a subset of trials to estimate channels with sentence-specific responses and test it in held-out data. Premotor and sensorimotor cortex of the left hemisphere indicated sentence-specific representations during perception and silent rehearsal of the target sentence. However, these sentence-specific representations were not identical across perception and rehearsal, indicating that the perception pattern transformed to distinct sentence-specific representations during mental rehearsal. In contrast, higher-order representations in prefrontal and temporoparietal cortices were shared across perception and rehearsal. In agreement with our initial hypothesis, the temporal activation patterns in prefrontal cortex were sensitive to the prior context information. Together, the data suggest “core” speech rehearsal areas may implement a sensorimotor transformation in support of verbal short-term memory, while more distributed networks, sensitive to semantics, expressed a shared pattern of activity, bridging perception and rehearsal.
Publications
- (2016). How long is now? The multiple timescales of language processing. Behavioral and Brain Sciences 39:e77
Honey CJ, Chen J, Müsch K, & Hasson U
(See online at https://doi.org/10.1017/S0140525X15000825) - (2017). Neural representations of perceived and remembered spoken sentences. Annual Meeting of the Society for Neuroscience, Washington, DC
Müsch K, Himberger K, Tan KM, Valiante TA, Honey CJ
- (2018). Causal Evidence for a Neural Component of Spatially Global Hemodynamic Signals. Neuron 97(4):734-736
Müsch K, Honey CJ
(See online at https://doi.org/10.1016/j.neuron.2018.02.003) - (2018). Transformation of speech sequences in human sensorimotor circuits
Müsch K, Himberger K, Tan KM, Valiante TA, Honey CJ
(See online at https://doi.org/10.1101/419358)
