Spoken lAnguage in motions: Learning and Adaptation of speech coMMunication in the context of BOdy motions

• Applicant: Dr. Susanne Fuchs
• Subject Area: General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
• Term: from 2018 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 390200946

Final Report Abstract

Who hasn't experienced this yet: you get a call from a friend who is on his way to a meeting. Although you can't see him, you can clearly hear on the phone that he is walking fast. In the Salammbo project we, i.e., researchers from the movement sciences, psychology, and lin guistics from Germany and France, are investigating the influence of movement on spoken language production. Specifically, we assume that breathing plays a mediating role between movement and spoken language because it is important for both processes: under physical exertion, there is an increased demand for oxygen intake and the exhaled air is needed to produce spoken language. The first goal of Salammbo is to create novel multimodal corpora with simultaneous record ings of movement, breath, and acoustic data of spoken language using state-of-the-art tech nology. For the first corpus, 25 German native speakers were recorded three times within a week or maximally 10 days. They were presented with stories including new names via video and asked to recall and retell the details of the stories under different movement conditions (without movement, with free hands, rhythmic movements with arms or legs). In the second corpus, we varied the physical effort and speaking tasks. Forty-eight female subjects were included. They rode bicycles under light and moderate exertion on a stationary ergometer and produced sustained vowels, read a text, or spontaneous speech. As a control condition, the same speech tasks were included without exercise. In different working programs, we investigated: a) the effects of physical exertion on respira tory, laryngeal, and supralaryngeal properties of spoken language, b) the relationship be tween movement speed and speech rate under different exertion levels, c) the influence of acceleration maxima on local acoustic parameters such as fundamental frequency and in tensity envelopes in different movement conditions, and d) the stability and flexibility of indi vidual breathing in different tasks. The results collected show a) a clear influence on all levels of spoken language studied, b) no linear interplay between movement and speech rate, c) an influence of acceleration max ima on the peaks of intensity (but not f0) when a certain threshold of acceleration is exceed ed, and d) the stability of individual breathing for specific tasks. These results suggest a complex interplay between movement-breathing-speech. They are considered empirical evi dence that models of speech production should also include whole-body movement and are incomplete so far. With our work, we close a research gap.

Publications

• Speech production and perception: Learning and memory. Series: Speech production and perception 6. Berlin: Peter Lang.
  Fuchs, Susanne; Joanne Cleland; & Amelie Rochet-Capellan
  
• Speaking while moving: Does the head compensate for the hands not being able to move? Proceedings of the 12th International Seminar on Speech Production. Providence, RI, USA.
  Serré et al.
  
• Speech during light and moderate physical activity: Effect on f0 and vocal intensity. Proceedings of the 12th International Seminar on Speech Production. Providence, RI, USA.
  Weston, H., Fuchs, S. & Rochet-Capellan, A.
  
• Changes in glottal source parameter values with light to moderate physical load. Proceedings of Interspeech, Brno, Czechia.
  Weston, H., Koenig, L.L. & Fuchs, S
  
• Speech breathing: variable but individual over time and according to limb movements. Annals of the New York Academy of Sciences, 1505(1), 142-155.
  Serré, Hélène; Dohen, Marion; Fuchs, Susanne; Gerber, Silvain; & Rochet‐Capellan, Amélie
  
• The respiratory foundations of spoken language. Annual Review of Linguistics, 7, 13-30.
  Fuchs, Susanne; & Rochet-Capellan, Amélie
  
• Leg movements affect speech intensity. Journal of Neurophysiology, 128(5), 1106-1116.
  Serré, Hélène; Dohen, Marion; Fuchs, Susanne; Gerber, Silvain; & Rochet-Capellan, Amélie
  
• Origins of vocal-entangled gesture. Neuroscience & Biobehavioral Reviews, 104836.
  Pouw, Wim; & Fuchs, Susanne
  
• Speech-breathing-limb interaction and coordination during the production and the memorization of verbal information.
  Serré, Hélène
  
• Speech breathing during physical activity is opportunistic. In: Radek Skarnitzl & Jan Volín (Eds.), Proceedings of the 20th International Congress of Phonetic Sciences (pp. 1082–106). Prague. Guarant International.
  Weston, H.
  
• On the relation between leg-motion rate and speech tempo during submaximal cycling exercise Journal of Speech, Language and Hearing Research
  Weston, Heather; Pouw, Wim; & Fuchs, Susanne