Cross-modal integration of language and face cues in German and French infants
Final Report Abstract
It was the aim of our CrossMod ANR-DFG project to determine when the binding between auditory and visual facial cues emerges during infancy, how language-specific it is and whether it narrows down to the native language toward the end of the first year. The originally planned experiments were successfully carried out and have already resulted in a number of publications. We have examined the processing of audio-visual fluent speech information by German and French infants, with four series of experiments: (i) the first part of the project has shown that infants at 3.5 months of age cannot discriminate between French and German based on visual cues only, but that 6-month-old German infants can. Surprisingly, however, German 6-month-olds were shown to prefer looking at the French stimuli, while French 6-month-olds did not show any preference when presentation times were short (30 sec) but also present a preference toward the French stimuli when presentation times were prolonged (60 sec). (ii) Then we tested audio-visual integration of speech and showed that German infants as young as 4.5 months of age can cross-modally match audiovisual fluent speech of their native and non-native languages, and that this capacity narrows down to the infants’ native language by the age of 6 months. French infants tested at 3.5, 6, and 12 months were not found to show a multisensory response, which raises a number of questions on language specific and stimulus-specific effects on the development of audio-visual integration. (iii) The third part of the project tested the role of prosodic cues on the perception of audio-visual speech. Kubicek et al. (2014) compared infants’ response to infant-directed (ID) vs. adult-directed (AD) speech and showed that 12-month-old infants were not capable of audio-visual integration with AD speech whereas they were with ID speech. In another series of experiments, it was showed that audio-visual integration is hampered when the two languages tested are from the same rhythmic class (here: German and English, both stress-timed). Both results prove that infants rely on salient prosodic cues for audio-visual integration, and suggest that their audio-visual integration is mainly driven by rhythmic cues. (iv) Finally, we observed the effect of visual and audio-visual cues on face scanning by 12-month-old infants, and showed that for the native language audio-visual information induced more scanning of the eyes than visual information only, which draws more attention to the mouth. For the non-native language, infants looked more at the mouth region. In sum, the CrossMod project provided a number of original results on audio-visual integration of speech in infants.
Publications
-
(2013). Cross-modal matching of audiovisual German and French speech in infancy. Biennial Meeting of the Society for Research in Child Development, Seattle, Washington, USA, April 18-20 2013, Abstract 169
Kubicek, C., Hillairet de Boisferon, A., Dupierrix, E., Pascalis, O., Loevenbruck, H., Gervain, J., & Schwarzer, G.
-
(2013). Face-scanning behavior to silently-talking faces in 12-month-old infants: The impact of pre-exposed auditory speech. International Journal of Behavioral Development, 37(2), 106-110
Kubicek, C., Hillairet de Boisferon, A., Dupierrix, E., Loevenbruck, H., Gervain, J., & Schwarzer, G.
-
(2013). Integration gehörter und gesehener Sprache im Säuglingsalter. Fachgruppentagung Entwicklungspsychologie der DGPs (EPSY) meeting, Saarbrücken, Germany, 9-11 September 2013, Poster 162
Kubicek, C., Hillairet de Boisferon, A., Dupierrix, E., Pascalis, O., Gervain, J., Loevenbruck, H., & Schwarzer, G.
-
(2014). Cross-modal matching of audio-visual German and French fluent speech in infancy. PLoS ONE 9(2): e89275
Kubicek, C., Hillairet de Boisferon, A., Dupierrix, E., Pascalis, O., Loevenbruck, H., Gervain, J., & Schwarzer, G.
-
(2014). The influence of infant-directed speech on 12-month-olds’ intersensory perception of fluent speech. Infant Behavior & Development, 37 (4),644-651
Kubicek, C., Gervain, J., Loevenbruck, H., Pascalis, O., & Schwarzer, G.
