TRR 62:  Companion-Technology for Cognitive Technical Systems

• Subject Area: Computer Science, Systems and Electrical Engineering ; Construction Engineering and Architecture ; Medicine
• Term: from 2009 to 2017
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 54371073

• Final Report Year: 2018

Final Report Abstract

The Transregional Collaborative Research Centre CRC/TRR 62 was driven by the vision to create technical systems of any kind as Companion systems – cognitive-technical systems, which adapt their functionality to each user individually. A Companion system’s behavior is guided by the user’s abilities, preferences, demands, and current needs, adapts to the current situation, disposition, and emotional state, and is always available, cooperative, and trustworthy, and is a competent and collaborative service provider. The guiding idea was the Companion-ability of technical systems. It manifests itself in the traits competence, individuality, adaptivity, cooperativeness, and trustworthiness. These Companion traits have been the subject of interdisciplinary research aiming to provide a technology that allows to create technical systems of every kind as Companion systems. In doing so, two different perspectives were considered. The system’s point of view was concerned with developing comprehensive cognitive abilities for technical systems. A well-orchestrated and effective interoperation of perception-, interaction-, planning-, and reasoning-processes creates these Companion traits and elevate technical systems to Companion systems. The user’s point of view investigated human-computer interaction and Companion traits from a psychological and neuro-biological perspective. Psychological models of behavior, new experimental paradigms, and the analysis of brain activity provided information on the effect of system behavior and thus enabled the effective implementation and evaluation of Companion traits. The most important results of the CRC/TRR 62 are • an interdisciplinary theory of Companion-abilities of cognitive technical systems; • an operational concept of Companion traits; • methods, models, tools, and components for realizing Companion traits in technical systems; • reference architectures for Companion systems; • prototypical Companion systems in the application domains Assistance for Installing a Complex Home Theater System, Worker Assistance in Car Manufacturing, as well as Assistance for Home Improvement Projects. The synergistically achieved results of interdisciplinary fundamental research within the CRC/TRR 62 serve a pivotal societal concern. Demands on individuals rise due to increased digitalization – the ubiquitous use of complex and versatile hard- and software systems in all areas of life. Simultaneously, new opportunities for supportive technology and digital assistance arise due to technological development. In this area of conflict, Companion technology constitutes a pioneering advancement: it enables the development of flexible, individually adaptive, truly user-friendly and competent (inter-)acting technical systems. Researchers in the CRC/TRR 62 have published more than 800 peer-reviewed publications, which have been published predominantly in high-ranking journals and conferences. The CRC’s comprehensive and interdisciplinary research results have furthermore been published in a dedicated summarizing book. This book with the title Companion Technology – A Paradigm Shift in Human-Technology Interaction (editor S. Biundo and A. Wendemuth) comprises 25 papers on 500 pages. It has been published in the series Cognitive Technologies of Springer.

Publications

• “Differential neuromodulation of acquisition and retrieval of avoidance learning by the lateral habenula and ventral tegmental area”. In: The Journal of Neuroscience: the Official Journal of the Society for Neuroscience 30.17 (2010). S. 5876–5883
  Shumake, Jason; Ilango, Anton; Scheich, Henning; Wetzel, Wolfram & Ohl, Frank W.
  
• “Memory Capacities for Synaptic and Structural Plasticity”. In: Neural Computation 22.2 (2010). S. 289–341
  Knoblauch, Andreas; Palm, Günther & Sommer, Friedrich T.
  
• “Semi-supervised learning for tree-structured ensembles of RBF networks with Co-Training”. In: Neural Networks 23.4 (2010). S. 497–509
  Abdel Hady, Mohamed Farouk; Schwenker, Friedhelm & Palm, Günther
  
• “Advanced User Assistance Based on AI Planning”. In: Cognitive Systems Research, Special Issue on Complex Cognition 12.3-4 (2011). S. 219–236
  Biundo, Susanne; Bercher, Pascal; Geier, Thomas; Müller, Felix & Schattenberg, Bernd
  
• “Altered Brain Activity During Emotional Empathy in Somatoform Disorder”. In: Human Brain Mapping 33 (2012). S. 2666–2685
  de Greck, Moritz; Scheidt, Lisa; Bölter, Annette F.; Frommer, Jörg; Ulrich, Cornelia; Stockum, Eva; Enzi, Björn; Tempelmann, Claus; Hoffmann, Thilo; Han, Shihui & Northoff, Georg
  
• “Personal Projectors for Pervasive Computing”. In: IEEE Pervasive Computing 11.2 (2012). S. 30–37
  Rukzio, Enrico; Holleis, Paul & Gellersen, Hans
  
• “Stereo-Camera-based Urban Environment Perception using Occupancy Grid and Object Tracking”. In: IEEE Transactions on Intelligent Transportation Systems (2011). S. 154–165
  Nguyen, Thien-Nghia; Michaelis, Bernd; Al-Hamadi, Ayoub; Tornow, Michael & Meinecke, Marc-Michael
  
• “Analysis of significant dialog events in realistic human–computer interaction”. In: Journal on Multimodal User Interfaces 8.1 (2014). S. 75–86
  Prylipko, Dmytro; Rösner, Dietmar; Siegert, Ingo; Günther, Stephan; Friesen, Rafael; Haase, Matthias; Vlasenko, Bogdan & Wendemuth, Andreas
  
• “Feedback that confirms reward expectation triggers auditory cortex activity”. In: Journal of Neurophysiology 110.8 (2013). S. 1860–1868
  Weis, Tina; Brechmann, André; Puschmann, Sebastian & Thiel, Christiane M.
  
• “Hierarchical Constraints – Providing Structural Bias for Hierarchical Clustering”. In: Machine Learning 94.3 (2014). S. 371–399
  Bade, Korinna & Nürnberger, Andreas
  
• “Real-Time Multi-Object Tracking using Random Finite Sets”. In: IEEE Transactions on Aerospace and Electronic Systems 49.4 (2013). S. 2666–2678
  Reuter, Stephan; Wilking, Benjamin; Wiest, Jurgen; Munz, Michael & Dietmayer, Klaus
  
• “Transsituational Individual-Specific Biopsychological Classification of Emotions”. In: IEEE Transactions on Systems, Man, and Cybernetics: Systems 43.4 (2013). S. 988–995
  Walter, Steffen; Kim, Jonghwa; Hrabal, David; Crawcour, Stephen Clive; Kessler, Henrik & Traue, Harald C.
  
• “Using unlabeled data to improve classification of emotional states in human computer interaction”. In: Journal on Multimodal User Interfaces 8.1 (2013). S. 5–16
  Schels, Martin; Kächele, Markus; Glodek, Michael; Hrabal, David; Walter, Steffen & Schwenker, Friedhelm
  
• “Comparative Learning Applied to Intensity Rating of Facial Expressions of Pain”. In: International Journal of Pattern Recognition and Artificial Intelligence 28 (5 2014). S. 168–186
  WERNER, PHILIPP; AL-HAMADI, AYOUB & NIESE, ROBERT
  
• “Computing with a canonical neural circuits model with pool normalization and modulating feedback”. In: Neural Computation 26.12 (2014). S. 2735–2789
  Brosch, Tobias & Neumann, Heiko
  
• “Crowd behaviour analysis and anomaly detection by statistical modelling of flow patterns”. In: International Journal of Data Mining, Modelling and Management 6 (2 2014). S. 168–186
  Pathan, Saira Saleem; Hamadi, Ayoub Al & Michaelis, Bernd
  
• “Graph Clusterings with Overlaps: Adapted Quality Indices and a Generation Model”. In: Neurocomputing 123 (2014). S. 13–22
  Gossen, Tatiana; Kotzyba, Michael & Nürnberger, Andreas
  
• “Learning Long-term Dependencies in Segmented- Memory Recurrent Neural Networks with Backpropagation of Error”. In: Neurocomputing 141 (2014). S. 54–64
  Glüge, Stefan; Böck, Ronald; Palm, Günther & Wendemuth, Andreas
  
• “Managing adaptive spoken dialogue for Intelligent Environments”. In: Journal of Ambient Intelligence and Smart Environments 6.5 (2014). S. 523–539
  Ultes, Stefan & Minker, Wolfgang
  
• “Plan, Repair, Execute, Explain - How Planning Helps to Assemble your Home Theater”. In: Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS 2014). AAAI Press, 2014, S. 386–394
  Bercher, Pascal; Biundo, Susanne; Geier, Thomas; Hoernle, Thilo; Nothdurft, Florian; Richter, Felix & Schattenberg, Bernd
  
• “Reasoning with Nominal Schemas through Absorption”. In: Journal of Automated Reasoning 53.4 (2014). S. 351–405
  Steigmiller, Andreas; Glimm, Birte & Liebig, Thorsten
  
• “The Labeled Multi-Bernoulli Filter”. In: IEEE Transactions on Signal Processing 62.12 (2014). S. 3246–3260
  Reuter, Stephan; Vo, Ba-Tuong; Vo, Ba-Ngu & Dietmayer, Klaus
  
• “Auditory intensity processing: Categorization versus comparison”. In: NeuroImage 119 (2015), S. 362–370
  Angenstein, Nicole & Brechmann, André
  
• “Coherence Across Components in Cognitive Systems – One Ontology to Rule Them All”. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI 2015). AAAI Press, 2015, S. 1442–1449
  G. Behnke, D. Ponomaryov, M. Schiller, P. Bercher, F. Nothdurft, B. Glimm und S. Biundo
  
• “Companion-Technology for Cognitive Technical Systems”. In: Künstliche Intelligenz, Special Issue on Companion Technologies 30.1 (2016). S. 71–75
  Biundo, Susanne & Wendemuth, Andreas
  
• “Fusion paradigms in cognitive technical systems for human-computer interaction”. In: Neurocomputing 161 (2015). S. 17–37
  Glodek, Michael; Honold, Frank; Geier, Thomas; Krell, Gerald; Nothdurft, Florian; Reuter, Stephan; Schüssel, Felix; Hörnle, Thilo; Dietmayer, Klaus; Minker, Wolfgang; Biundo, Susanne; Weber, Michael; Palm, Günther & Schwenker, Friedhelm
  
• “On event-based optical flow detection”. In: Frontiers in Neuroscience 9.137 (2015). S. 1–15
  Brosch, Tobias; Tschechne, Stephan & Neumann, Heiko
  
• “OPEN_EmoRec_II – A Multimodal Corpus of Human-Computer Interaction”. In: International Journal of Computer, Electrical, Automation, Control and Information Engineering 9.5 (2015), S. 1181–1187
  S. Rukavina, S. Gruss, S. Walter, H. Hoffmann und H. C. Traue
  
• “Pay-As-You-Go Description Logic Reasoning by Coupling Tableau and Saturation Procedures”. In: Journal of Artificial Intelligence Research 54 (2015). S. 535–592
  Steigmiller, Andreas & Glimm, Birte
  
• “The complex duration perception of emotional faces: Effects of face direction”. In: Frontiers in Psychology: Emotion Science 6.262 (2015). S. 1–10
  Kliegl, Katrin M.; Limbrecht-Ecklundt, Kerstin; Dürr, Lea; Traue, Harald C. & Huckauf, Anke
  
• “Assessing the Expressivity of Planning Formalisms through the Comparison to Formal Languages”. In: Proceedings of the 26th International Conference on Automated Planning and Scheduling (ICAPS 2016). AAAI Press, 2016, S. 158–165
  Höller, Daniel; Behnke, Gregor; Bercher, Pascal & Biundo, Susanne
  
• “Counting votes in coupled decisions”. In: Theory and Decision 80 (2016). S. 1–42
  Wendemuth, Andreas & Simonelli, Italo
  
• “Effects of Neutral and Fearful Mood on Duration Estimation of Neutral and Fearful Face Stimuli”. In: Timing & Time Perception 4.1 (2016). S. 30–47
  Eberhardt, Lisa V.; Huckauf, Anke & Kliegl, Katrin M.
  
• “Facial Expression Reactions to Feedback in a Human-Computer Interaction—Does Gender Matter?” In: Psychology 7.3 (2016). S. 356–367
  Rukavina, Stefanie; Gruss, Sascha; Hoffmann, Holger & Traue, Harald C.
  
• “Lend a Hand to Service Robots: Overcoming System Limitations by Asking Humans”. In: Dialogues with Social Robots: Enablements, Analyses, and Evaluation. Hrsg. von K. Jokinen. Bd. 427. Lecture Notes in Electrical Engineering. Springer Singapore, 2016, S. 321–329
  Schüssel, Felix; Walch, Marcel; Rogers, Katja; Honold, Frank & Weber, Michael
  
• “Methods for Person-Centered Continuous Pain Intensity Assessment from Bio-Physiological Channels”. In: IEEE Journal of Selected Topics in Signal Processing (2016). S. 854–864
  Kachele, Markus; Thiam, Patrick; Amirian, Mohammadreza; Schwenker, Friedhelm & Palm, Gunther
  
• “Neural Information Processing in Cognition: We Start to Understand the Orchestra, but Where is the Conductor?” In: Frontiers in Computational Neuroscience 10.3 (2016). S. 1–6
  Palm, Günther
  
• “Pupil Size Changes as an Active Information Channel for Biofeedback Applications”. In: Applied Psychophysiology and Biofeedback 41.3 (2016). S. 331–339
  Ehlers, Jan; Strauch, Christoph; Georgi, Juliane & Huckauf, Anke
  
• Companion Technology: A Paradigm Shift in Human-Technology Interaction. 1. Aufl. Cognitive Technologies. Cham: Springer International Publishing, 2017
  S. Biundo und A. Wendemuth, Hrsg.
  
• “Effect of sequential comparison on active processing of sound duration”. In: Human Brain Mapping 38.9 (2017). S. 4459–4469
  Angenstein, Nicole & Brechmann, André