Harmonic Artificial Intelligence based on Linear Operators

• Applicant: Dr. Felix Dietrich
• Subject Area: Computer Architecture, Embedded and Massively Parallel Systems ; Mathematics
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 468830823

Simulating complex, highly interconnected systems such as the climate, biology, or society typically involve methods from the "traditional" field of scientific computing. These methods are usually reliable and explainable through their foundations in rigorous mathematics. Examples are efficient space discretization schemes such as sparse grids and scalable, parallel solvers for partial differential equations. Unfortunately, most of them are not immediately applicable to the extremely high-dimensional, heterogeneous, and scattered data where neural networks are usually used. Those methods employed in the AI community, on the other hand, are typically not reliable or explainable in the traditional sense, and pose problems illustrated through adversarial examples and brittle generalization results.Toward the goal of explainable, reliable, and efficient AI, I propose to connect the two worlds of scientific HPC and deep learning, forming a concept I call "Harmonic AI". Specifically, I will combine linear operator theory and deep learning methods through harmonic analysis. The benefit of a link between AI and linear operators is bi-directional. Inference, classification, and training of neural networks will be understood mostly in terms of linear algebra. This will open the field to much more mathematical rigor and enable more mathematicians to work on AI methods. Simultaneously, applied AI researchers obtain reliable methods that can currently only be found for problems outside the field, such as Finite Element Methods or iterative Newton-Raphson solvers.The first three years of the project are devoted to explainability of AI by bridging the gap to rigorous mathematics: Leveraging the common principles between the Laplace operator, Gaussian processes, and neural networks, Harmonic AI will connect AI and linear algebra. This will allow me to explain unsupervised data representation algorithms. The second core objective in this first phase is to bridge the theory between the linear Koopman operator and deep neural networks. I will bring ideas from dynamical systems theory to the AI community, to explain the layered neural networks and stochastic optimization algorithms in terms of dynamical evolution operators.The core objective in the second phase (years four to six) is to devise robust and reliable numerical algorithms harnessing the connection between linear operators and neural networks from phase A. The algorithms will be integrated in an HPC environment. I will release a free and open source software for Harmonic AI to make its algorithms available to a broad research community. Throughout the project, to disseminate, demonstrate, and test the new methods in a proof of concept, I will collaborate with the simulation groups from C. Mendl studying quantum dynamics and from G. Köster studying human crowds.

• DFG Programme: Independent Junior Research Groups