TRR 62:  Companion-Technology for Cognitive Technical Systems

• Subject Area: Computer Science, Systems and Electrical Engineering ; Construction Engineering and Architecture ; Medicine
• Term: from 2009 to 2017
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 54371073

• Final Report Year: 2018

Final Report Abstract

The Transregional Collaborative Research Centre CRC/TRR 62 was driven by the vision to create technical systems of any kind as Companion systems – cognitive-technical systems, which adapt their functionality to each user individually. A Companion system’s behavior is guided by the user’s abilities, preferences, demands, and current needs, adapts to the current situation, disposition, and emotional state, and is always available, cooperative, and trustworthy, and is a competent and collaborative service provider. The guiding idea was the Companion-ability of technical systems. It manifests itself in the traits competence, individuality, adaptivity, cooperativeness, and trustworthiness. These Companion traits have been the subject of interdisciplinary research aiming to provide a technology that allows to create technical systems of every kind as Companion systems. In doing so, two different perspectives were considered. The system’s point of view was concerned with developing comprehensive cognitive abilities for technical systems. A well-orchestrated and effective interoperation of perception-, interaction-, planning-, and reasoning-processes creates these Companion traits and elevate technical systems to Companion systems. The user’s point of view investigated human-computer interaction and Companion traits from a psychological and neuro-biological perspective. Psychological models of behavior, new experimental paradigms, and the analysis of brain activity provided information on the effect of system behavior and thus enabled the effective implementation and evaluation of Companion traits. The most important results of the CRC/TRR 62 are • an interdisciplinary theory of Companion-abilities of cognitive technical systems; • an operational concept of Companion traits; • methods, models, tools, and components for realizing Companion traits in technical systems; • reference architectures for Companion systems; • prototypical Companion systems in the application domains Assistance for Installing a Complex Home Theater System, Worker Assistance in Car Manufacturing, as well as Assistance for Home Improvement Projects. The synergistically achieved results of interdisciplinary fundamental research within the CRC/TRR 62 serve a pivotal societal concern. Demands on individuals rise due to increased digitalization – the ubiquitous use of complex and versatile hard- and software systems in all areas of life. Simultaneously, new opportunities for supportive technology and digital assistance arise due to technological development. In this area of conflict, Companion technology constitutes a pioneering advancement: it enables the development of flexible, individually adaptive, truly user-friendly and competent (inter-)acting technical systems. Researchers in the CRC/TRR 62 have published more than 800 peer-reviewed publications, which have been published predominantly in high-ranking journals and conferences. The CRC’s comprehensive and interdisciplinary research results have furthermore been published in a dedicated summarizing book. This book with the title Companion Technology – A Paradigm Shift in Human-Technology Interaction (editor S. Biundo and A. Wendemuth) comprises 25 papers on 500 pages. It has been published in the series Cognitive Technologies of Springer.

Publications

• “Differential neuromodulation of acquisition and retrieval of avoidance learning by the lateral habenula and ventral tegmental area”. In: The Journal of Neuroscience: the Official Journal of the Society for Neuroscience 30.17 (2010). S. 5876–5883
  J. Shumake, A. Ilango, H. Scheich, W. Wetzel und F. W. Ohl
  (See online at https://doi.org/10.1523/jneurosci.3604-09.2010)
• “Memory Capacities for Synaptic and Structural Plasticity”. In: Neural Computation 22.2 (2010). S. 289–341
  A. Knoblauch, G. Palm und F. T. Sommer
  (See online at https://doi.org/10.1162/neco.2009.08-07-588)
• “Semi-supervised learning for tree-structured ensembles of RBF networks with Co-Training”. In: Neural Networks 23.4 (2010). S. 497–509
  M. F. A. Hady, F. Schwenker und G. Palm
  (See online at https://doi.org/10.1016/j.neunet.2009.09.001)
• “Advanced User Assistance Based on AI Planning”. In: Cognitive Systems Research, Special Issue on Complex Cognition 12.3-4 (2011). S. 219–236
  S. Biundo, P. Bercher, T. Geier, F. Müller und B. Schattenberg
  (See online at https://doi.org/10.1016/j.cogsys.2010.12.005)
• “Stereo-Camera-based Urban Environment Perception using Occupancy Grid and Object Tracking”. In: IEEE Transactions on Intelligent Transportation Systems (2011). S. 154–165
  T. Nguyen, B. Michaelis, A. Al-Hamadi, M. Tornow und M.-M. Meinecke
  (See online at https://doi.org/10.1109/TITS.2011.2165705)
• “Altered Brain Activity During Emotional Empathy in Somatoform Disorder”. In: Human Brain Mapping 33 (2012). S. 2666–2685
  M. De Greck, L. Scheidt, A. Bölter, J. Frommer, C. Ulrich, E. Stockum, B. Enzi, C. Tempelmann, T. Hoffmann, S. Han und G. Northoff
  (See online at https://doi.org/10.1002/hbm.21392)
• “Personal Projectors for Pervasive Computing”. In: IEEE Pervasive Computing 11.2 (2012). S. 30–37
  E. Rukzio, P. Holleis und H. Gellersen
  (See online at https://doi.org/10.1109/MPRV.2011.17)
• “Feedback that confirms reward expectation triggers auditory cortex activity”. In: Journal of Neurophysiology 110.8 (2013). S. 1860–1868
  T. Weis, A. Brechmann, S. Puschmann und C. Thiel
  (See online at https://doi.org/10.1152/jn.00128.2013)
• “Real-Time Multi-Object Tracking using Random Finite Sets”. In: IEEE Transactions on Aerospace and Electronic Systems 49.4 (2013). S. 2666–2678
  S. Reuter, B. Wilking, J. Wiest, M. Munz und K. Dietmayer
  (See online at https://doi.org/10.1109/TAES.2013.6621844)
• “Transsituational Individual-Specific Biopsychological Classification of Emotions”. In: IEEE Transactions on Systems, Man, and Cybernetics: Systems 43.4 (2013). S. 988–995
  S. Walter, J. Kim, D. Hrabal, S. C. Crawcour, H. Kessler und H. C. Traue
  (See online at https://doi.org/10.1109/TSMCA.2012.2216869)
• “Using unlabeled data to improve classification of emotional states in human computer interaction”. In: Journal on Multimodal User Interfaces 8.1 (2013). S. 5–16
  M. Schels, M. Kächele, M. Glodek, D. Hrabal, S. Walter und F. Schwenker
  (See online at https://doi.org/10.1007/s12193-013-0133-0)
• “Analysis of significant dialog events in realistic human–computer interaction”. In: Journal on Multimodal User Interfaces 8.1 (2014). S. 75–86
  D. Prylipko, D. Rösner, I. Siegert, S. Günther, R. Friesen, M.-t. Haase, B. Vlasenko und A. Wendemuth
  (See online at https://doi.org/10.1007/s12193-013-0144-x)
• “Comparative Learning Applied to Intensity Rating of Facial Expressions of Pain”. In: International Journal of Pattern Recognition and Artificial Intelligence 28 (5 2014). S. 168–186
  P. Werner, A. Al-Hamadi und R. Niese
  (See online at https://doi.org/10.1142/S0218001414510082)
• “Computing with a canonical neural circuits model with pool normalization and modulating feedback”. In: Neural Computation 26.12 (2014). S. 2735–2789
  T. Brosch und H. Neumann
  (See online at https://doi.org/10.1162/neco_a_00675)
• “Crowd behaviour analysis and anomaly detection by statistical modelling of flow patterns”. In: International Journal of Data Mining, Modelling and Management 6 (2 2014). S. 168–186
  S. Pathan, A. Al-Hamadi und B. Michaelis
  (See online at https://doi.org/10.1504/IJDMMM.2014.063196)
• “Graph Clusterings with Overlaps: Adapted Quality Indices and a Generation Model”. In: Neurocomputing 123 (2014). S. 13–22
  T. Gossen, M. Kotzyba und A. Nürnberger
  (See online at https://doi.org/10.1016/j.neucom.2012.09.046)
• “Hierarchical Constraints – Providing Structural Bias for Hierarchical Clustering”. In: Machine Learning 94.3 (2014). S. 371–399
  K. Bade und A. Nürnberger
  (See online at https://doi.org/10.1007/s10994-013-5397-9)
• “Learning Long-term Dependencies in Segmented- Memory Recurrent Neural Networks with Backpropagation of Error”. In: Neurocomputing 141 (2014). S. 54–64
  S. Glüge, R. Böck, G. Palm und A. Wendemuth
  (See online at https://doi.org/10.1016/j.neucom.2013.11.043)
• “Managing adaptive spoken dialogue for Intelligent Environments”. In: Journal of Ambient Intelligence and Smart Environments 6.5 (2014). S. 523–539
  S. Ultes und W. Minker
  (See online at https://doi.org/10.3233/AIS-140275)
• “Plan, Repair, Execute, Explain - How Planning Helps to Assemble your Home Theater”. In: Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS 2014). AAAI Press, 2014, S. 386–394
  P. Bercher, S. Biundo, T. Geier, T. Hoernle, F. Nothdurft, F. Richter und B. Schattenberg
  (See online at https://doi.org/10.1609/icaps.v24i1.13664)
• “Reasoning with Nominal Schemas through Absorption”. In: Journal of Automated Reasoning 53.4 (2014). S. 351–405
  A. Steigmiller, B. Glimm und T. Liebig
  (See online at https://doi.org/10.1007/s10817-014-9310-4)
• “The Labeled Multi-Bernoulli Filter”. In: IEEE Transactions on Signal Processing 62.12 (2014). S. 3246–3260
  S. Reuter, B.-T. Vo, B.-N. Vo und K. Dietmayer
  (See online at https://doi.org/10.1109/TSP.2014.2323064)
• “Auditory intensity processing: Categorization versus comparison”. In: NeuroImage 119 (2015), S. 362–370
  N. Angenstein und A. Brechmann
  (See online at https://doi.org/10.1016/j.neuroimage.2015.06.074)
• “Coherence Across Components in Cognitive Systems – One Ontology to Rule Them All”. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI 2015). AAAI Press, 2015, S. 1442–1449
  G. Behnke, D. Ponomaryov, M. Schiller, P. Bercher, F. Nothdurft, B. Glimm und S. Biundo
  (See online at https://dl.acm.org/doi/10.5555/2832415.2832450)
• “Fusion paradigms in cognitive technical systems for human-computer interaction”. In: Neurocomputing 161 (2015). S. 17–37
  M. Glodek, F. Honold, T. Geier, G. Krell, F. Nothdurft, S. Reuter, F. Schüssel, T. Hörnle, K. Dietmayer, W. Minker, S. Biundo, M. Weber, G. Palm und F. Schwenker
  (See online at https://doi.org/10.1016/j.neucom.2015.01.076)
• “On event-based optical flow detection”. In: Frontiers in Neuroscience 9.137 (2015). S. 1–15
  T. Brosch, S. Tschechne und H. Neumann
  (See online at https://doi.org/10.3389/fnins.2015.00137)
• “OPEN_EmoRec_II – A Multimodal Corpus of Human-Computer Interaction”. In: International Journal of Computer, Electrical, Automation, Control and Information Engineering 9.5 (2015), S. 1181–1187
  S. Rukavina, S. Gruss, S. Walter, H. Hoffmann und H. C. Traue
  (See online at https://doi.org/10.5281/zenodo.1338277)
• “Pay-As-You-Go Description Logic Reasoning by Coupling Tableau and Saturation Procedures”. In: Journal of Artificial Intelligence Research 54 (2015). S. 535–592
  A. Steigmiller und B. Glimm
  (See online at https://doi.org/10.1613/jair.4897)
• “The complex duration perception of emotional faces: Effects of face direction”. In: Frontiers in Psychology: Emotion Science 6.262 (2015). S. 1–10
  K. M. Kliegl, K. Limbrecht-Ecklundt, L. Dürr, H. C. Traue und A. Huckauf
  (See online at https://doi.org/10.3389/fpsyg.2015.00262)
• “Assessing the Expressivity of Planning Formalisms through the Comparison to Formal Languages”. In: Proceedings of the 26th International Conference on Automated Planning and Scheduling (ICAPS 2016). AAAI Press, 2016, S. 158–165
  D. Höller, G. Behnke, P. Bercher und S. Biundo
  (See online at https://doi.org/10.1609/icaps.v26i1.13758)
• “Companion-Technology for Cognitive Technical Systems”. In: Künstliche Intelligenz, Special Issue on Companion Technologies 30.1 (2016). S. 71–75
  S. Biundo und A. Wendemuth
  (See online at https://doi.org/10.1007/s13218-015-0414-8)
• “Counting votes in coupled decisions”. In: Theory and Decision 80 (2016). S. 1–42
  A. Wendemuth und I. Simonelli
  (See online at https://doi.org/10.1007/s11238-015-9532-x)
• “Effects of Neutral and Fearful Mood on Duration Estimation of Neutral and Fearful Face Stimuli”. In: Timing & Time Perception 4.1 (2016). S. 30–47
  L. V. Eberhardt, A. Huckauf und K. M. Kliegl
  (See online at https://doi.org/10.1163/22134468-00002060)
• “Facial Expression Reactions to Feedback in a Human-Computer Interaction—Does Gender Matter?” In: Psychology 7.3 (2016). S. 356–367
  S. Rukavina, S. Gruss, H. Hoffmann und H. C. Traue
  (See online at https://doi.org/10.4236/psych.2016.73038)
• “Lend a Hand to Service Robots: Overcoming System Limitations by Asking Humans”. In: Dialogues with Social Robots: Enablements, Analyses, and Evaluation. Hrsg. von K. Jokinen. Bd. 427. Lecture Notes in Electrical Engineering. Springer Singapore, 2016, S. 321–329
  F. Schüssel, M. Walch, K. Rogers, F. Honold und M. Weber
  (See online at https://doi.org/10.1007/978-981-10-2585-3_26)
• “Methods for Person-Centered Continuous Pain Intensity Assessment from Bio-Physiological Channels”. In: IEEE Journal of Selected Topics in Signal Processing (2016). S. 854–864
  M. Kächele, P. Thiam, M. Amirian, F. Schwenker und G. Palm
  (See online at https://doi.org/10.1109/JSTSP.2016.2535962)
• “Neural Information Processing in Cognition: We Start to Understand the Orchestra, but Where is the Conductor?” In: Frontiers in Computational Neuroscience 10.3 (2016). S. 1–6
  G. Palm
  (See online at https://doi.org/10.3389/fncom.2016.00003)
• “Pupil Size Changes as an Active Information Channel for Biofeedback Applications”. In: Applied Psychophysiology and Biofeedback 41.3 (2016). S. 331–339
  J. Ehlers, C. Strauch, J. Georgi und A. Huckauf
  (See online at https://doi.org/10.1007/s10484-016-9335-z)
• Companion Technology: A Paradigm Shift in Human-Technology Interaction. 1. Aufl. Cognitive Technologies. Cham: Springer International Publishing, 2017
  S. Biundo und A. Wendemuth, Hrsg.
  (See online at https://doi.org/10.1007/978-3-319-43665-4)
• “Effect of sequential comparison on active processing of sound duration”. In: Human Brain Mapping 38.9 (2017). S. 4459–4469
  N. Angenstein und A. Brechmann
  (See online at https://doi.org/10.1002/hbm.23673)