SFB 673:  Alignment in Communication

• Subject Area: Humanities ; Computer Science, Systems and Electrical Engineering ; Social and Behavioural Sciences
• Term: from 2006 to 2015
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 13226603

• Final Report Year: 2015

Final Report Abstract

The CRC 673 “Alignment in Communication” was founded through two sources of inspiration. The first was work from the previous CRC 360 “Situated Artificial Communicators”, in which linguists, psychologists, and computer scientists at Bielefeld University had joined forces to investigate how artifi-cial agents can exploit contextual (situational) information from different modalities in order to support cooperation and improve the efficiency and robustness of human-agent communication. The second source of inspiration was the emergence of a new and revolutionary approach to human-human communication called “Interactive Alignment” (Pickering & Garrod 2004). The Interactive Align-ment approach was a reaction to the then-fashionable paradigms for cognitive models of human-human communication and the corresponding technical implementations of human-machine communication. These paradigms were based primarily on classical generative linguistics and traditional Artificial Intelli-gence approaches. Both relied on complex reasoning, involving slow and arduous symbolic computa-tions. This led to theories and implementations of communicative systems that were inflexible, brittle, and inefficient. Human communication, on the other hand, is flexible, robust, and highly efficient. Most striking-ly, human communicators do not appear to rely on explicit negotiations to check on mutual convergence. Interactive Alignment theory offered an elegant account of how to bridge the gap between current theo-ries and observed phenomena. It provocatively suggested that we do not need the computational mecha-nisms traditionally assumed to be essential for achieving effective communication. Instead, human com-municators can rely on a combination of situational knowledge, routinization and resource-efficient ‘prim-ing’ processes in order to automatically achieve rapid and successful convergence (alignment) of their respective situation models. At the time of the founding of the CRC, the theory of interactive alignment was new and rather specula-tive. Therefore, in the first funding phase, the CRC focused on validating the concept of interactive align-ment and on extending interactive alignment theory to cover a broader range of communicative phenom-ena and cognitive processes that underlie agent-agent communication. The combined use of experimen-tation, corpus analysis, and simulative-experimental loops led to a more accurate appreciation of the ex-planatory power of interactive alignment and to a better understanding of the limitations of the theory. This phase has yielded a number of important insights into the nature and scope of the interactive align-ment as an explanatory device. Experimental work further validated interactive alignment by confirming the presence of priming effects in many different representational domains. In addition, the implementa-tion and simulation of alignment-based processing in artificial agents demonstrated the validity of the concept in application contexts. We also extended the concept of interactive alignment, having discov-ered that interactive alignment based on priming of representations is (in empirical reality) and needs to be (in technical implementations) augmented by additional forms of processing, such as “anti-alignment”, ”asymmetrical” alignment, and strategic alignment. Inspired by the exciting results from the first funding phase, the second phase addressed a number of hitherto understudied aspects of interactive alignment. One important extension of the research ad-dressed the role of timing in alignment. This research produced new insights regarding interpersonal temporal alignment at different representationals levels through anticipation and entrainment. Temporal alignment is also central to intrapersonal alignment in the case of multimodal utterances, e.g., involving both speech and gesture. Also studied in the second phase was how alignment at one representational level relates to alignment at other levels. Several projects investigated “vertical” aspects of alignment, focussing on the interfaces between syntax and semantics, and between semantics and pragmatics. For greater ecological validity, the second funding phase also included the study of natural data to comple-ment and validate the experimental studies. The technical projects focussed on systemic aspects of alignment, demonstrating that interactive alignment can be used to increase the intuitiveness, efficiency, and acceptance of human-machine interaction. It also showed that alignment principles can be applied to a broader range of contextual aspects of communication, such as space, attention, emotions, or collabo-rative action. This has created a new impetus for applying alignment principles in complex information processing systems to facilitate efficient cooperation using ‘informationally narrow’ coupling, i.e. without explicit exchange of exhaustive state information.

Publications

• (2010). Modeling the production of coverbal iconic gestures by learning Bayesian Decision Networks. Applied Artificial Intelligence, 24(6), 530-551
  Bergmann, K., & Kopp, S.
  
• (2011). How Can I Help? - Spatial Attention Strategies for a Receptionist Robot. International Journal of Social Robotics, 3(4), 383-393
  Holthaus, P., Pitsch, K., & Wachsmuth, S.
  
• (2011). Integrating feature maps and competitive layer architectures for motion segmentation. Neurocomputing, 74(9), 1372-1381
  Steffen, J., Pardowitz, M., Steil, J.J., & Ritter, H.
  
• (2012). A model of intentional communication: AIRBUS (Asymmetric Intention Recognition with Bayesian Updating of Signals). In S. Brown-Schmidt, J. Ginzburg, & S. Larsson (Eds.), Proceedings of SemDial 2012 (SeineDial) – 16th Workshop on the Semantics and Pragmatics of Dialogue. Paris: Université Paris-Diderot, 149-150
  De Ruiter, J.P., & Cummins, C.
  
• (2012). Contextually enriched argument linking. In R. Finkbeiner, J. Meibauer, & P. Schumacher (Eds.), What is a Context? Amsterdam: John Benjamins, 199-228
  Klein, U.
  
• (2012). Exploiting spatial descriptions in visual scene analysis. Cognitive Processing, 13(1), 369-374
  Ziegler, L., Johannsen, K., Swadzba, A., De Ruiter, J.P., & Wachsmuth, S.
  
• (2012). Kommunikation im 21. Jahrhundert: Alter Dialog-Wein in neuen Technik- Schläuchen. Zeitschrift für Literaturwissenschaft und Linguistik, 42, 13-27
  De Ruiter, J.P.
  
• (2012). Rapid entrainment to spontaneous speech: A comparison of oscillator models. In N. Miyake, D. Peebles & R. P. Cooper (Eds.), Proceedings of the 34th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society, 1721-1726
  Inden, B., Malisz, Z., Wagner, P., & Wachsmuth, I.
  
• (2012). Towards an ontology of categories for multimodal annotation. In Proceedings of the Workshop "Describing Language Resources with Metadata" at Language Resources and Evaluation Conference (LREC), 49-54
  Menke, P., & Cimiano, P.
  
• (2013). A computational model of cooperative spatial behaviour for virtual humans. In T. Tenbrink, J. Wiener, & C. Claramunt (Eds.), Representing Space in Cognition: Interrelations of Behaviour, Language, and Formal Models. Oxford: Oxford University Press, 147-168
  Nguyen, N., & Wachsmuth, I.
  
• (2013). Alignment in Communication: Towards a New Theory of Communication (Advances in Interaction Studies, 6). Amsterdam: Benjamins
  Wachsmuth, I., De Ruiter, J.P., Jaecks, P., & Kopp, S. (Eds.)
  
• (2013). Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications. Journal on Multimodal User Interfaces, 7(1-2), 5-18
  Lücking, A., Bergmann, K., Hahn, F., Kopp, S., & Rieser, H.
  
• (2013). Effects of speaker emotional facial expression and listener age on incremental sentence processing. PLOS ONE, 8(9)
  Carminati, M.N., & Knoeferle, P.
  
• (2013). Interaction phonology – a temporal coordination component enabling representational alignment within a model of communication. In I. Wachsmuth, J.P. De Ruiter, P. Jaecks, & S. Kopp (Eds.), Advances in Interaction Studies: Vol. 6. Alignment in Communication: Towards a New Theory of Communication. Amsterdam: Benjamins. 109-132
  Wagner, P., Malisz, Z., Inden, B., & Wachsmuth, I.
  
• (2013). MExiCo: A Library for Managing Multimodal Data Collections. Procedia – Social and Behavioral Sciences, 95, 105-110
  Menke, P., & Cimiano, P.
  
• (2013). Modeling the semantic coordination of speech and gesture under cognitive and linguistic constraints. In: Intelligent Virtual Agents, LNAI 8108. Berlin, Heidelberg: Springer, 203-216
  Bergmann, K., Kahl, S., & Kopp, S.
  
• (2013). On grounding natural kind terms in humanrobot communication. KI - Künstliche Intelligenz, 27, 107-118
  Peltason, J., Rieser, H., Wachsmuth, S., & Wrede, B.
  
• (2013). Priming and conceptual pacts in overhearers’ adoption of referring expressions. In M. Knauff, M. Pauen, N. Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society, 1869-1874
  Behnel, M., Cummins, C., Sichelschmidt, L., & De Ruiter, J.P.
  
• (2013). Realtime 3D segmentation for Human-Robot Interaction. Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), Tokyo, Japan
  Ückermann, A., Haschke, R., & Ritter, H.
  
• (2013). Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent. In J. Epps, F. Chen, S. Oviatt, K. Mase, A. Sears, K. Jokinen, & B. Schuller (Eds.), Proceedings of the 15th International Conference on Multimodal Interaction. New York: ACM
  Inden, B., Malisz, Z., Wagner, P., & Wachsmuth, I.
  
• (2013). What is the link between emotional and communicative alignment in interaction? In I. Wachsmuth, J. de Ruiter, P. Jaecks, & S. Kopp (Eds.), Advances in Interaction Studies: Vol. 6. Alignment in Communication: Towards a New Theory of Communication. Amsterdam: Benjamins, 205-224
  Jaecks, P., Damm, O., Hielscher-Fastabend, M., Malchus, K., Stenneken, P., & Wrede, B.
  
• (2013): Empathy and its modulation in a virtual human. In I.J. Timm & M. Thimm (Eds.), LNAI: Vol. 8077. KI 2013: Advances in Artificial Intelligence. Berlin, Heidelberg: Springer, 25-36
  Boukricha, H., Wachsmuth, I., Carminati, M. N., & Knoeferle, P.
  
• Optimal Constructions. In G. Legendre, M. Putnam, & E. Zaroukian (Eds.), Advances in Optimality theoretic-syntax and semantics. Studies in Theoretical Linguistics. Oxford: Oxford University Press
  Vogel R.
  
• (2014). A detailed analysis of a new 3D Spatial Feature Vector for indoor scene classification. Robotics and Autonomous Systems, 62(5), 646–662
  Swadzba, A., & Wachsmuth, S.
  
• (2014). Children’s Syntactic-Priming Magnitude: Lexical Factors and Participant Characteristics. Journal of Child Language, 1-14. Available on CJO 2014
  Foltz, A., Thiele, K., Kahsnitz, D., & Stenneken, P.
  
• (2014). Communicating emotions - A model for natural emotions in HRI. Proceedings of the second international conference on Human-agent interaction - HAI, 269-272
  Damm, O., &, Wrede, B.
  
• (2014). Computational Approaches to the Pragmatics Problem. Language and Linguistics Compass, 8(4), 133-143
  Cummins, C., & De Ruiter, J.P.
  
• (2014). Influences of semantic and syntactic incongruence on readiness potential in turn-end anticipation Frontiers in Human Neuroscience, 8(296), 1-9
  Wesselmeier, H., Jansen, S., & Müller, H.M.
  
• (2014). Notes on Disagreement. In D. Gutzman, J. Köpping, & C. Meier (Eds.), Evaluations - Denotations - Entities. Studies in Context, Contents and the Foundation of Semantics. Leiden: Brill, 276-305
  Kracht M, & Klein U.
  
• (2014). Perceptual grouping through competition in coupled oscillator networks. Neurocomputing, 141, 76-83
  Meier, M., Haschke, R., & Ritter, H.
  
• (2014). Spatial references with gaze and pointing in shared space of humans and robots. In C. Freksa, B. Nebel, M. Hegarty, & T. Barkowsky (Eds.), Lecture Notes in Computer Science: Vol. 8684. Spatial Cognition IX. Berlin [u.a.]: Springer, 121-136
  Renner, P., Pfeiffer, T., & Wachsmuth, I.
  
• (2014). Using the readiness potential of button-press and verbal response within spoken language processing. Journal of Neuroscience Methods, 232, 24-29
  Jansen, S., Wesselmeier, H., De Ruiter, J.P. & Müller, H.M.
  
• (2015). Anticipation in turn-taking: mechanisms and information sources. Frontiers in Psychology, 6(89)
  Riest, C., Jorschick, A., & De Ruiter, J.P.
  
• (2015). Exploring the alignment space – lexical and gestural alignment with real and virtual humans. Frontiers in ICT 2:7
  Bergmann, K., Branigan, H.P., & Kopp, S.
  
• (2015). Lexical alignment in triadic communication. Frontiers in Psychology 6(127)
  Foltz, A., Gaspers, J., Thiele, K., Stenneken, P., & Cimiano, P.
  
• (2015). Pointing and reference reconsidered. Journal of Pragmatics, 77, 56-79
  Lücking, A., Pfeiffer, T., & Rieser, H.
  
• (2015). Temporal Effects of Alignment in Text-based, Task-oriented Discourse. Discourse Processes, 1-33
  Foltz, A., Gaspers, J., Meyer, C., Thiele, K., Cimiano, P., & Stenneken, P.