The sounds of meaning: Investigating the functional interaction between action sounds and semantics

• Applicant: Professor Dr. Friedemann Pulvermüller
• Subject Area: General, Cognitive and Mathematical Psychology ; General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
• Term: from 2016 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 316928999

• Final Report Year: 2022

Final Report Abstract

The SOM project addressed the mechanisms of semantic understanding of sounds and words in and out of sentence contexts at the linguistic experimental level. By comparing behavioral and brain responses to semantically isolated and context-embedded congruent and incongruent, we obtained clues about the neural mechanisms underlying semantic representation. These could be fruitfully applied to validate current theories of semantic processing, in particular the ASS vs APS families of models. Results showed a clear superiority of APS theory. Unexpected was the discovery of a novel brain component which emerged in anticipation of predictable stimuli, the Prediction Potential. This new brain index reveals the subjects’ semantic and pronological predictions before the predicted stimuli actually appear. The observation of systematic functional relationship between the PP and the subsequent brain responses, such as the MMN and the N400, opens a novel avenue for testing theories of predictive coding.

Publications

• (2016). Somatotopic Semantic Priming and Prediction in the Motor System. Cereb Cortex
  Grisoni, L., Dreyer, F. R., & Pulvermüller, F.
  (See online at https://doi.org/10.1093/cercor/bhw026)
• (2017). Neural correlates of semantic prediction and resolution in sentence processing. J Neurosci.
  Grisoni, L., Miller, T. M., & Pulvermüller, F.
  (See online at https://doi.org/10.1523/jneurosci.2800-16.2017)
• (2019). Prediction and Mismatch Negativity Responses Reflect Impairments in Action Semantic Processing in Adults With Autism Spectrum Disorders. Front. Hum. Neurosci.
  Grisoni, L., Moseley, R.L., Motlagh, S., Kandia, D., Sener, N., Pulvermüller, F., Roepke, S., Mohr, B.
  (See online at https://doi.org/10.3389/fnhum.2019.00395)
• (2019). Prediction mechanisms in motor and auditory areas and their role in sound perception and language understanding. Neuroimage, 199, 206-216
  Grisoni, L., Mohr, B., & Pulvermuller, F.
  (See online at https://doi.org/10.1016/j.neuroimage.2019.05.071)
• (2020). Semantic Prediction in Brain and Mind. Trends Cogn Sci.
  Pulvermüller, F., & Grisoni, L.
  (See online at https://doi.org/10.1016/j.tics.2020.07.002)
• (2021). Correlated Brain Indexes of Semantic Prediction and Prediction Error: Brain Localization and Category Specificity. Cereb Cortex
  Grisoni, L., Tomasello, R., & Pulvermüller, F.
  (See online at https://doi.org/10.1093/cercor/bhaa308)