Adapting Foundation Models for Simulating Physical Processes

• Applicants: Professor Mathias Niepert, Ph.D.; Professor Dr. Nils Thuerey
• Subject Area: Methods in Artificial Intelligence and Machine Learning
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 569019417

Computational modeling plays a crucial role in solving complex mathematical problems across disciplines such as physics, the natural sciences, and engineering, with applications ranging from fluid to molecular simulations. Scientific machine learning has emerged as a powerful paradigm that integrates traditional scientific computing with modern machine learning, enabling the development of new computational models with improved accuracy and efficiency. Recent advances in foundation models (FMs)—large neural networks trained in a self-supervised fashion on massive datasets—have demonstrated remarkable generalization in domains like language and vision. However, adapting FMs to physical systems presents unique challenges, including a lack of standardized datasets, high computational costs of data generation, and the difficulty of incorporating physics-informed inductive biases. Moreover, the training and use of FMs are often computationally expensive, limiting their practical utility for efficient simulations. This project aims to address these challenges through four key research questions: (Q1) identifying suitable inductive biases and training methods for probabilistic surrogate models, (Q2) developing uncertainty-driven active learning strategies for cost-efficient fine-tuning, (Q3) distilling knowledge from FMs into efficient, physics-aware models, and (Q4) curating and generating meaningful data for FM training with minimal resource usage. To answer these questions, the project pursues three objectives: (1) develop resource-efficient adaptations of FMs through active learning and distillation, (2) create probabilistic adaptations for uncertainty-aware modeling, and (3) incorporate physics-awareness into FMs to enhance robustness and generalization. The research will focus on fluid and fluid-like PDE systems as test cases for evaluating effectiveness, scalability, and scientific impact.

• DFG Programme: Research Grants