Stability, Accuracy and Efficiency in Hybrid Finite Element / Neural Network Simulations

• Applicants: Professor Dr. Christian Lessig; Professor Dr. Thomas Richter
• Subject Area: Mathematics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 537063406

The solution of partial differential equations is a central subject of numerical analysis and an indispensable tool in science and engineering. Existing approaches, such as finite elements, can provide solutions efficiently and robustly in many applications. Deep neural networks emerged in the last few years as an alternative approach with promising results. Techniques that are completely or partially based on neural networks, however, currently lack the mathematical guarantees and insights available for established approaches. Their relative performance and practical robustness in applications is also unclear at the moment. We will work towards a mathematical theory of numerical techniques that combine finite elements and deep neural networks for the solution of partial differential equations. Our hypothesis is that such a hybrid approach can provide a computationally more efficient and more accurate solution than either approach alone. We consider the Navier-Stokes equations with the neural networks representing fine scale behavior not resolved by finite elements. The networks are trained using high-resolution reference data. We will therefore not pursue physical or mathematical constraints on the solutions, as in PINNs, and consider it an important but orthogonal research direction. The objective of the proposed project is to develop mathematically rigorous analyses, but we consider it also as important to study the practicality of our results through implementations. A research code for hybrid fluid flow solvers will therefore be implemented and made publicly available. We build on recent work that showed that the mathematical analysis of deep neural networks is possible using tools developed for the analysis of finite element methods. We will extend these results to hybrid numerical time stepping schemes for the Navier-Stokes equations and consider practically relevant setups. Further we extend existing results to state-of-the-art neural network architectures, e.g. transformers. These are one of the most powerful architectures used in practice and at the same time well suited for scientific computing and a mathematical analysis. Central questions we will address are stability and accuracy of the hybrid simulations, i.e. that they remain bounded and that a neural network is able to improve the accuracy. For a hybrid solver, this requires, among other things, neural networks that are stable for admissible inputs but also a coupling to the finite element part that preserves stability. Second, we will explore adaptive solution schemes where a posteriori or neural network-based error estimates are used to refine a solution if necessary, to meet predefined error criteria. We believe that the results obtained in the proposed project will also be of relevance for a more complete theory for neural network-based simulations.

• DFG Programme: Research Grants