Multidimensional analysis of simulated conversations

• Applicant: Professor Dr.-Ing. Sebastian Möller
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Communication Technology and Networks, High-Frequency Technology and Photonic Systems, Signal Processing and Machine Learning for Information Technology
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 493809605

In the preceding research project (project no. 320253669), a simulation of conversations was developed that modeled turn taking and speech intelligibility in the presence of packet loss and transmission delays. The resulting conversations were used to predict overall conversation quality. However, conversation quality can be divided into perceptual dimensions of the listening, speaking, and interaction phases of a conversation, to provide a deeper understanding of the quality assessment and potential clues to the cause of the degradations. ITU-T Recommendation P.804 standardizes the diagnostic testing procedure to capture the three conversational phases with their perceptual dimensions. It is the aim of this research project to simulate the listening, speaking and interaction phases according to ITU-T Recommendation P.804 and to predict the perceptual dimensions using the resulting data. For this purpose, the test protocol of the three conversation phases is implemented in the simulation environment. For the simulation of the speech phase, models are developed for the detection of the feedback of own speech (echo) and also for the adaptation of the speaking behavior in the presence of this degradation. For the prediction of the dimensions of the listening phase, the model P.AMD, which is in the process of standardization, can be used. For the prediction of the perceptual dimensions of the speaking and interaction phases, new models are being developed and evaluated. As a result of this research project, a new simulation as well as a quality prediction model for impaired conversations are expected. The model and the simulation should be promoted as an international standard, and be transferred to the industry partner.

• DFG Programme: Research Grants (Transfer Project)
• Application Partner: Rohde & Schwarz SwissQual AG