Assisted-scoring of free-text tasks through linguistic normalization of answers

• Applicants: Professorin Dr. Andrea Horbach; Professor Dr.-Ing. Torsten Zesch
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Applied Linguistics, Computational Linguistics
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 563947383

In addition to imparting knowledge, checking learning success is one of the most important tasks in any teaching-learning process. Free-text tasks, i.e. tasks that require a short, freely formulated text as an answer, play a central role. Due to the variance of learner responses, fully automated assessment using machine learning methods has so far only been possible to a very limited extent. In particular, the lack of transparency of the models stand in the way of practical application. We therefore propose an alternative paradigm for the use of artificial intelligence and natural language processing to support scoring, in which the actual scoring itself is not automated (with all the associated acceptance problems and ethical challenges), but where the scoring is still carried out by humans with the aid of text normalization methods. In this way, we are aiming for easier adaptation to new tasks and, at the same time, a significant reduction in the workload of teachers with time-consuming assessment tasks. In order to achieve these goals, a central focus of the project is on the further development of text normalization procedures so that the linguistic variance of the answers can be significantly reduced. Another focus is the practical integration into eAssessment systems and the implementation of usage studies under realistic conditions. In particular, we investigate how graders interact with the system and how our approach influences the accuracy and time required for the assessments compared to conventional human assessment on the one hand and a fully automated assessment model on the other. The research community will benefit from the project in many ways (in addition to the main research results): An annotated dataset will be created for research into text normalization processes, which will be freely available beyond the project and will stimulate further research into the underlying linguistic phenomena. The text normalization data will be used to organize shared tasks, thus stimulating further research in this field. All tools developed, in particular a prototype for assisted scoring with text normalization, will also be made available as open-source projects.

• DFG Programme: Research Grants