Project Details
Projekt
Attention in Large Language Models: Linguistic Grounding, Cognitive Modeling, and Social Application
Subject Area
Applied Linguistics, Computational Linguistics
Term
since 2026
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 579383584
Large language models (LLMs) have transformed natural language processing, yet the mechanisms of their success remain only partly understood. A key component is the attention mechanism, which encodes contextual dependencies between words and often reveals linguistically meaningful patterns. Its value as an explanatory tool for cognitive and psycholinguistic processes, however, is contested. Missing are methodological standards that specify when attention can be treated as interpretable evidence, how such evidence can be developed into cognitively and linguistically grounded concepts, and how these can be operationalized into metrics to support transparent, safe applications, such as improving scientific and medical communication. This project tests the hypothesis that the attention mechanism in LLMs can serve as a computational analogue of human text comprehension. We pursue three objectives: (O1) grounding attention in cognitive, and psycholinguistic correlates of text comprehension (foundational understanding), (O2) applying data-driven discovery to derive attention-based metrics and evaluate their predictive validity against behavioral and neural correlates (robust assessment), and (O3) applying discovered metrics to assess and personalize the comprehensibility of scientific and medical text in a transparent and interpretable manner (safe applicability). Together, these objectives connect to the pillars of the LaSTing Priority Programme. To achieve this, Subproject 1 derives a theory-driven word facilitation metric from attention distributions in the transformer architecture and tests its predictive power beyond traditional linguistic metrics. Subproject 2 applies equation discovery to explore the broader hypothesis space of attention-derived metrics, identifying novel predictors of behavioral and neural correlates of text comprehension. Subproject 3 expands these metrics to personalized comprehension profiles to improve individual understanding of scientific and medical texts. Preliminary results show that attention-based facilitation explains unique variance in text comprehension, especially for long and rare words that traditional metrics misclassify as difficult. Building on this, the project will deliver reproducible pipelines for discovering and validating psycholinguistic metrics grounded in LLM architectures and practical tools for model-driven personalization of scientific and medical texts. Beyond immediate outcomes, it contributes to a broader vision of LLMs as scientific instruments for explaining and improving human text comprehension, advancing both the methodological toolkit of the cognitive language sciences and the societal impact of empirically grounded LLM-based metrics.
DFG Programme
Priority Programmes
International Connection
USA
Co-Investigators
Professorin Dr. Christiane Maaß; Dr. Roland Mühlenbernd
Cooperation Partner
Professor Lawrence David, Ph.D.
