Digital Product - Digital User (DPDU)
Final Report Abstract
The project aimed at improving current state-of-the-art cognitive modelling regarding spatial interaction with technical systems and basic neuroscience research on spatial abilities. It enables both project partners to gain valuable insights into specific questions of the respective research fields while allowing for synergistic effects that provide insights above and beyond what would be possible with isolated biopsychological or cognitive modelling methods. In this project the department of Cognitive Modelling in Dynamic Human-Machine Systems was collaborating with the Department of Biopsychology and Neuroergonomics. The main aim was to learn more about human spatial skills and processes by using and combining different methodologies and study paradigm. These methodologies are cognitive modelling, EEG methods, behavioral experimental data, human-system interaction studies. In a series of 4 experiments, test persons and cognitive models were analyzed with regard to their interaction with virtual and real products. The main component of 3 experiments is the ability to mentally rotate and mentally fold in set ups in front of a monitor. Mental rotation describes the ability to rotate a depicted object in the mind. Mental folding is the idea of folding objects such as cube nets together in one's mind. Two images are normally shown for this purpose. After mental testing, a statement should be made as to whether the two objects shown are identical. In the fourth experimental set up participants had to interact with a virtual operating table and a real operation table, manipulating it to specific configurations that make folding and rotating of mental representations necessary. The first and second experiment was focused to either mental folding or rotation on the monitor. The third experiment combines the two methods. The fourth experiment uses the product example of an operating table. A key component is the communication between the Adaptive Control of Thought-Rational (ACT-R for short) model, the digital user, and the digital product, i.e. the CAD image of the real product. EEG data was measured to obtain e.g. brain dynamics and source localization for the specific temporal patterns and gained deeper understanding about specific structures responsible for different forms of spatial transformation. Through cognitive modelling, assumptions about the cognitive processes involved in mental spatial transformation derived from theories of spatial cognitive mechanisms in cognitive science research were tested and validated on the experiment results. These assumptions include a formula predicting the duration of spatial transformative processes, but also involvement of visual encoding, spatial representation, and memory processes. On this basis, the cognitive models showed high prediction accuracy for reaction times and learning behavior of test participants. Furthermore, intra-trial processes suggested by the models were compared to EEG recorded during the experiments, and provided insight on spatial cognition, namely the influence of task difficulty, the identification of involved brain areas, and the different demands of spatial representation and transformation by task modality. Furthermore, a ready-to-use production pipeline for connecting cognitive models to their digital twin was developed.
Publications
-
Kognitives Modell zu combined Mental Rotation & Folding
Preuss, Kai
-
Kognitives Modell zu Mental Folding
Preuss, Kai
-
Kognitives Modell zu Mental Rotation
Preuss, Kai
-
Mental Rotation and Mental Folding online experiment implementation
Hilton, Christopher
-
Verallgemeinertes räumliches Modul für kognitive Modellierung in ACT-R
Preuss, Kai
-
Verhaltensdaten "Interaktion mit Medizintechnik"
Preuss, Kai
-
An implementation of Universal Spatial Transformative Cognition in ACT-R. In Proceedings of the 17th International Conference on Cognitive Modelling.
Preuss, Kai.; Raddatz, Leonie & Russwinkel, Nele
-
Proposing a Cognitive System for Universal Mental Spatial Transformations. Poster presented at the 41st Annual Conference of the Cognitive Science Society. Montreal, Quebec, Canada.
Preuss, Kai & Russwinkel, Nele
-
Cognitive Processing Stages During Mental Folding Are Reflected in Eye Movements. ACM Symposium on Eye Tracking Research and Applications, 1-5. ACM.
Preuss, Kai; Hilton, Christopher; Gramann, Klaus & Russwinkel, Nele
-
Workload-over-time modeling in a cognitive architecture. Presentation during the Mental Effort Workshop as part of the 42nd Annual Meeting of the Cognitive Science Society.
Russwinkel, Nele & Preuss, Kai
-
Cognitive Modelling of a Mental Rotation Task Using a Generalized Spatial Framework. In Proceedings of the 19th International Conference on Cognitive Modelling.
Preuss, Kai & Russwinkel, Nele
-
Lessons Learned From Modelling Situated Cognitive Agents Interacting With a Dynamic Environment. At the 19th International Conference on Cognitive
Seiling, L.; Plitt, M.; Schweidler, P.; Klückmann, O.; Befort, L.; Konczir, R.; Reuter, S.; Preuss, K.; Wiese, S. & Russwinkel, N.
-
Verhaltensdaten und EEG-Daten Mental Rotation & Mental Folding
Hilton, Christopher
-
A general spatial transformation process? Assessing the neurophysiological evidence on the similarity of mental rotation and folding. Neuroimage: Reports, 2(2), 100092.
Hilton, Christopher; Raddatz, Leonie & Gramann, Klaus
-
Combining mental rotation and mental folding: Using EEG to understand spatial transformation mechanisms. Talk given at 64th Tagung experimentell arbeitender Psycholog:innen
Hilton, Christopher & Gramann, Klaus
-
Creating cognitive models on mental spatial transformation. Presentation held at the Doctoral Symposium of the German Society for Cognitive Science 2022.
Preuss, Kai
-
Using Cognitive Models and EEG Data To Investigate Spatial Cognition. Poster presented at the 15th Biannual Conference of the German Society for Cognitive Science.
Preuss, Kai & Russwinkel, Nele
-
Cognitive processing stages in mental rotation – How can cognitive modelling inform HsMM-EEG models?. Neuropsychologia, 188, 108615.
Heimisch, Linda; Preuss, Kai & Russwinkel, Nele
-
HsMM-EEG Daten und Analyse zu Mental Rotation für Heimisch et al.
Heimisch, Linda
-
kognitive Simulationsdaten und Analyse zu Mental Rotation für Heimisch et al.
Preuss, Kai
-
Statistik zum Abgleich von Cluster ERPs mit kognitiven Modellen zu Mental Rotation & Mental Folding für Preuss et al.
Preuß, Kai & Hilton, Christopher
-
Towards A Human-Centered Digital Twin. Procedia CIRP, 118, 324-329.
Preuss, Kai; Schulte, Svenja Nicole; Rzazonka, Lukas; Befort, Lilian; Fresemann, Carina; Stark, Rainer & Russwinkel, Nele
