Learning from Humans – Building for Humans
Human Cognitive and Systems Neuroscience
Final Report Abstract
A holistic approach is important for the design of Human-Cyber-Physical Systems (HCPS), which involves the collaboration between humans and machines. While the technical viability of these systems has garnered attention in the past, human supervision, support, and intervention remain crucial. The goal of this project between the Technical University of Munich, University of Oldenburg and the DLR was to develop co-operative HCPS, where humans and machines work together, leveraging each other’s strengths and compensating for their individual limitations. We focused on understanding three key factors: human perception and its limitations, human decision-making influenced by cognitive load, and trust in automation. To achieve these goals, highly synchronized experimental research was conducted in stateof-the-art simulation environments in the contributing labs. The research involved behavioral measurements, human state measurements, brain-sensing technology, human-machine interaction (HMI) engineering and various modeling approaches. Through cluster analysis, we were able to uncover naturally occurring driving strategies at intersections. Our results revealed that individuals with visual field loss tend to rely more heavily on extensive and early scanning as a compensatory strategy. We also discovered that the complexity of the driving situation significantly influences the effectiveness of compensatory strategies. In addition, we found that humans evaluate actions in the decisionmaking phase differently when they interact with an autonomous vehicle or a human driven vehicle. This was expressed in functional near-infrared spectroscopy (fNIRS) brain activation and partly in the behavioral tendencies. Moreover, we observed interactions between working memory load and decision-making at the brain-level. To better understand these interactions, we used a model-based approach and found that even simple control actions in driving likely share resources with working memory tasks. This highlights why cognitive load and driving performance may interact in some cases. Furthermore, we used the same model to make a priori predictions about interventions that manipulate cognitive load to avoid severe underload. We found that all interventions tested in a model introduce switching costs that assistive systems need to consider. Additionally, we developed an assistance concept designed to enhance scanning behaviors and facilitate hazard avoidance at intersections and on straight roads. With the early detection of safety-critical maneuvers advanced driver assistance systems could play a vital role in preventing accidents at intersections, ultimately leading to a decrease both in overall accident incidence and potentially even the number of fatal crashes. Our findings provide valuable insights for the development of these advanced systems and their potential impact on road safety.
Publications
-
Improving the Detection of User Uncertainty in Automated Overtaking Maneuvers by Combining Contextual, Physiological and Individualized User Data. Communications in Computer and Information Science, 390-397. Springer International Publishing.
Trende, Alexander; Hartwich, Franziska; Schmidt, Cornelia & Fränzle, Martin
-
A Causal Model of Intersection-Related Collisions for Drivers With and Without Visual Field Loss. Lecture Notes in Computer Science, 219-234. Springer International Publishing.
Biebl, Bianca; Kacianka, Severin; Unni, Anirudh; Trende, Alexander; Rieger, Jochem W.; Lüdtke, Andreas; Pretschner, Alexander & Bengler, Klaus
-
A comparative, sociotechnical design perspective on Responsible Innovation: multidisciplinary research and education on digitized energy and Automated Vehicles. Journal of Responsible Innovation, 8(3), 421-444.
Hess, David J.; Lee, Dasom; Biebl, Bianca; Fränzle, Martin; Lehnhoff, Sebastian; Neema, Himanshu; Niehaus, Jürgen; Pretschner, Alexander & Sztipanovits, Janos
-
Combining contextual and neurophysiological information for predicting driver’s turning intent. 3rd Neuroergonomics Conference on The Brain at Work and in everyday life, 11-16th September 2021.
Trende A., Unni A., Fränzle M. & Rieger J.W.
-
Criticality perception in dynamic traffic scenarios: an ACT-R model. Paper presented at Virtual MathPsych/ICCM 2021.
Földes-Cappellotto, N., Held, M., Baumann, M. & Stoll, T.
-
Decision making in human-autonomous vehicle interaction. Presented at the 3rd Neuroergonomics Conference on The Brain at Work and in everyday life, 11-16th September 2021.
Unni A., Trende A., Biebl B., Kacianka S., Lüdtke A., Bengler K., Pretschner A. & Rieger J.W.
-
Evaluation of graphical human-machine interfaces for turning manoeuvres in automated vehicles. 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 77-80. ACM.
Krefting, Ina; Trende, Alexander; Unni, Anirudh; Rieger, Jochem; Luedtke, Andreas & Fränzle, Martin
-
I Spy with My Mental Eye – Analyzing Compensatory Scanning in Drivers with Homonymous Visual Field Loss. Lecture Notes in Networks and Systems, 552-559. Springer International Publishing.
Biebl, Bianca & Bengler, Klaus
-
Modelling Turning Intention in Unsignalized Intersections with Bayesian Networks. Communications in Computer and Information Science, 289-296. Springer International Publishing.
Trende, Alexander; Unni, Anirudh; Rieger, Jochem & Fraenzle, Martin
-
Pupil size indicators of task interactions during driving. 3rd Neuroergonomics Conference on The Brain at Work and in everyday life, 11- 16th September 2021.
Arndt R., Kretzmeyer B., Unni A. & Rieger J.W.
-
Utilizing ACT-R to investigate interactions between working memory and visuospatial attention while driving [conference presentation]. Tagung für experimentell arbeitende Psychologen 2021, Ulm, Germany.
Held, M., Borst, J., Unni, A. & Rieger, J. W.
-
Utilizing ACT-R to investigate interactions between working memory and visuospatial attention while driving. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 43, No. 43).
Held, M., Borst, J., Unni, A. & Rieger, J.
-
Utilizing ACT-R to investigate interactions between working memory and visuospatial attention while driving. Paper presented at Virtual MathPsych/ICCM 2021.
Held, M., Borst, J., Unni, A. & Rieger, J. W.
-
A Case-Study for a Human-Centered Approach to Traffic Management Systems. Communications in Computer and Information Science, 259-266. Springer International Publishing.
Trende, Alexander; Krefting, Ina; Unni, Anirudh; Rieger, Jochem W. & Fränzle, Martin
-
Challenges in modeling situation awareness and criticality estimation in driving based on a video-study: An ACT-R model models [conference presentation]. Tagung für experimentell arbeitende Psychologen 2022, Cologne, Germany.
Földes, N., Held, M., Stoll, T. & Baumann, M.
-
Dealing with highly dynamic models: How behavioral and neuroimaging data can guide the modeling process in applied driving models [conference presentation]. Tagung für experimentell arbeitende Psychologen 2022, Cologne, Germany.
Held, M., Borst, J., Unni, A. & Rieger, J. W.
-
Driver's turning intent recognition model based on brain activation and contextual information. Frontiers in Neuroergonomics, 3.
Trende, Alexander; Unni, Anirudh; Jablonski, Mischa; Biebl, Bianca; Lüdtke, Andreas; Fränzle, Martin & Rieger, Jochem W.
-
Experimental validation of cognitive modeling. Tagung für experimentell arbeitende Psychologen 2022, Cologne, Germany.
Russwinkel, N., Rieger, J.W. & Ragni, M.
-
Investigating Differences in Behavior and Brain in Human-Human and Human-Autonomous Vehicle Interactions in Time-Critical Situations. Frontiers in Neuroergonomics, 3.
Unni, Anirudh; Trende, Alexander; Pauley, Claire; Weber, Lars; Biebl, Bianca; Kacianka, Severin; Lüdtke, Andreas; Bengler, Klaus; Pretschner, Alexander; Fränzle, Martin & Rieger, Jochem W.
-
Multitasking while driving: central bottleneck or problem state interference? Paper presented at In-Person MathPsych/ICCM 2022.
Held, M., Borst, J. & Rieger, J. W.
-
Multitasking While Driving: Central Bottleneck or Problem State Interference?. Human Factors: The Journal of the Human Factors and Ergonomics Society, 66(5), 1564-1582.
Held, Moritz; Rieger, Jochem W. & Borst, Jelmer P.
-
Opportunities and Limitations of a Gaze-Contingent Display to Simulate Visual Field Loss in Driving Simulator Studies. Frontiers in Neuroergonomics, 3.
Biebl, Bianca; Arcidiacono, Elena; Kacianka, Severin; Rieger, Jochem W. & Bengler, Klaus
-
A cognitive model testing different interventions to prevent harmful mind-wandering during driving [conference presentation]. Tagung für experimentell arbeitende Psychologen 2023, Trier, Germany.
Held, M., Borst, J., Unni, A. & Rieger, J. W.
-
Modeling and experimental validation in real-life environments. Tagung für experimentell arbeitende Psychologen 2023, Trier, Germany.
Held, M. & Rieger, J.W.
-
Modeling safety risks at intersections for Drivers with Limited Visual Perception [conference presentation]. Tagung für experimentell arbeitende Psychologen 2023, Trier, Germany.
Biebl, B. & Bengler, K.
-
The Real Sorting Hat – Identifying Driving and Scanning Strategies in Urban Intersections with Cluster Analysis. HCI 2023, July 23-28, Kopenhagen. . [Poster extended abstract accepted].
Biebl, B. & Bengler, K.
