Project Details
Projekt
Unreal engines — Understanding language models through resource-optimal analysis: Implicit Bayesian pragmatic reasoning & emergent causal world models
Applicant
Professor Dr. Michael Franke
Subject Area
Applied Linguistics, Computational Linguistics
General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
Theoretical Philosophy
General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
Theoretical Philosophy
Term
since 2026
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 579368432
This project combines insights from linguistic pragmatics, analytical philosophy and the literature on causal inference from statistics to investigate in how far language models can be genuinely said to understand language (in a human-like way). Aiming for generalizable, lasting results, the project aims for hard formal results and conceptual *in-principle* arguments. To that end, it proposes resource-optimal analysis which analyses language models as optimized solutions to next-token prediction given their architectural constraints. Adopting recent explications of the notion of “understanding” based on (causal) dependency models, the project addresses the leading research question after language understanding in language models as a question of causal latent recovery, as familiar from statistics and statistical machine learning. In particular, since recently successful models of probabilistic pragmatic reasoning equate pragmatic language understanding with Bayesian reasoning about the “language-generating process”, the project investigates under which conditions language models approximate such Bayesian pragmatic reasoning in human-like ways implicitly in their internal computations. Methodologically, the project utilizes formal modeling, mathematical proof (where possible), interventionist simulation studies, and and philosophical analysis.
DFG Programme
Priority Programmes
