A robust, reliable and multimodal AI system for pain quantification

• Applicants: Professor Dr.-Ing. Ayoub Al-Hamadi; Professor Dr. Steffen Walter
• Subject Area: Medical Informatics and Medical Bioinformatics
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 532219633

The project deals with the development of a robust, reliable and multimodal AI system for pain quantification based on the X-ITE database acquired in the previous project. The main focus is to improve the quality of treatment and the associated strengthening of the health and life quality of patients with dementia during the recovery and monitoring phase after surgical interventions. It also aims to enable better treatment of pain and its causes by supporting and relieving medical staff in pain assessment through automated real-time pain monitoring and by enabling more precise, individual and situation-specific analgesia. Especially in dementia patients, pain recognition is considerably more difficult, as the affected persons often forget their pain suffering or have lost the ability to verbally express it. Due to these cognitive limitations, external assessment instruments should be used for pain recognition for these patients, since self-report as the gold standard does not provide reliable information here. Regarding automatic pain recognition, various challenges arise in terms of data (objective recording of pain intensity, variations in appearance, facial expression, head pose, illumination, or partial occlusions) and algorithms (interpretation of facial expressions, limited availability of samples, choice of model, and unequal distribution of classes). These challenges are addressed in the following sub-objectives: 1. The use of deep neural networks and transfer learning with large existing in-the-wild databases to increase robustness to variances in appearance, illumination, etc., 2. evaluating previously unused modalities of body pose, thermal images, and side views of the head, as well as complex methods for fusing different modalities, 3. combining deep learning and temporal evaluation, for example, by using temporal convolutional networks (TCN) or transformers, and 4. investigating the achievable quality of measurement when sensing is limited to subsets of modalities (e.g., skin conductance only or noncontact sensing only). Continuous prediction of pain intensities will be enabled using regression. By using deep neural networks, transformers and Long Short-Term Memory (LSTM) in combination with transfer learning, as well as additional labels evaluating pain expressed by facial expressions, a significant improvement of pain recognition in terms of reliability, accuracy and robustness is expected, which should form the basis for transferring the system into clinical practice.

• DFG Programme: Research Grants