Chess Expertise from Eye Gaze and Emotion

• Applicant: Professor Dr. Thomas Schack
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2015 to 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 284240816

CEEGE is a multidisciplinary scientific research project. The primary impacts will be improved scientific understanding in the disciplines of Computer Science and Cognitive Neuro-Science. The aim of this project is to experimentally evaluate and compare current theories for mental modeling for problem solving and attention, as well as to refine and evaluate techniques for observing the physiological reactions of humans to situation that inspire pleasure, displeasure, arousal, dominance and fear. In this project, we will observe the visual attention, physiological responses and mental states of subject with different levels of expertise solving classic chess problems, and participating in chess matches. We will observe chess players using eye-tracking, sustained and instantaneous face-expressions (micro-expressions), skin conductivity, blood flow (BVP), respiration, posture and other information extracted from audio-visual recordings and sensor readings of players. We will use the recorded information to estimate the mental constructs with which the players understand the game situation. Information from visual attention as well as physiological reactions will be used to determine and model the degree to which a player understands the game situation in terms of abstract configurations of chess pieces. This will provide a structured environment that we will use for experimental evaluation of current theories of mental modeling and emotional response during problem solving and social interaction. The project is organized in three phases. During the first phase, we will observe individual players of different levels of chess expertise solving known chess problems. We will correlate scan-path from eye tracking and other information about visual attention to established configurations of pieces and known solutions to chess problems. This will allow us to construct a labeled corpus of chess play that can be used to evaluate competing techniques for estimating mental models and physiological responses. In a second phase, we will observe the attention and face expressions of pairs of players of different levels of chess ability during game play. In particular, we will seek to annotate and segment recordings with respect to the difficulty of the game situation as well as situations which elicit particularly strong physiological reactions. In the final phase, we will use these recordings to evaluate the effectiveness of competing techniques for mental modeling and observation of emotions in terms of their abilities to predict the chess abilities of players, game outcomes and individual moves and player self reports.

• DFG Programme: Research Grants
• International Connection: France
• Partner Organisation: Agence Nationale de la Recherche / The French National Research Agency
• Co-Investigator: Professor Dr. Kai Essig
• Cooperation Partner: Professor Dr. James Crowley