Bridging the skin: Facial surface and mimic muscles as a unit: Fully automated classification of motoric function and emotional expression in patients with facial palsy (BRIDGING THE GAP: MIMICS AND MUSCLES)

• Applicants: Professor Dr.-Ing. Joachim Denzler; Professor Dr. Orlando Guntinas-Lichius
• Subject Area: Otolaryngology, Phoniatrics and Audiology
• Term: from 2019 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 427899908

Facial palsy is the most common paresis or paralysis of a cranial nerve. A facial palsy results in most cases in a unilateral motoric dysfunction of the mimic muscles. For instance, incomplete eye closure or blinking is leading to a dry eye and dysfunction of the perioral muscles is leading to impaired intake of food and impaired speaking. The disturbed capability for emotional expressions is essential for the patients, for instance, because usual smiling is not possible anymore. Goal of optimal treatment is good rehabilitation of motoric function (standard goal in clinical routine) and of emotional expressiveness (hitherto often neglected). Standard in clinical routine to classify the severity of the disease as well as to monitor changes under therapy are mainly subjective and therefore unreliable and imprecise assessment tools. Aim of this proposal is to develop an objective measurement system using automated image analysis of the face for the description of the motoric disturbances and at the same time of the deficits in emotional expression. So far, most automated approaches for analysis of facial palsy were focused on static evaluations of 2D-datasets or evaluation of asymmetry on the surface of sequences of 3D-point clouds. To date, only a few concepts used accepted objective quantitative measures. The impact on the emotional expressiveness was not analyzed at all. In the present proposal standardized 3-D video recordings of mimic movements and emotional expressions of healthy subjects and patients will be used as training datasets. A muscle model will be included by fusion of measurements of electromyography recording of mimic muscles while performing standardized movements together with synchronous 3D-recordings of the face. Furthermore, data from sonography recordings of facial muscles will be included into the model as preexisting knowledge. The changes of emotional expressiveness and consequences on quality of life will be assessed by standard questionnaires and finally merged with the objective and automatically classified movement data to an overall index. Using groups of algorithms for unsupervised learning, so called generative adversarial networks, and synthetic accumulation of training data, it will be the first time to use deep learning approaches to improve the treatment of facial palsy. The findings gained by this project are not only relevant for facial palsy, but in general for any disease with disturbance of mimic muscle function or emotional expression. The results obtained herein for patients with facial palsy can serve as a model.

• DFG Programme: Research Grants
• Co-Investigator: Professor Dr. Christian Dobel