Assessment of Deep Learning through Meanfield Theory

• Applicant: Professor Dr. Michael Herty
• Subject Area: Mathematics
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 462234017

Kinetic and meanfield theory has proven an useful mathematical tool for hierarchical modeling of a variety of physical and sociological processes. It in particular allows to study emergent behavior as consequence of particle--to--particle dynamics. Modern learning methods can mathematically be reformulated such that an particle interaction structure emerges. E.g.~the class of residual deep neural networks in the limit of infinitely many layer leads to a coupled system of ordinary differential equations for the activation state of neurons. This arising system can be reformulated as an interacting 'particle' system where the state of each particle corresponds to the activation state of a neuron at a certain point in time. Within this proposal, we aim to exploit and extend existing meanfield theory to provide a mathematical framework for modern learning methods that allow for this description and hence focusing on learning by deep neural networks and learning through filtering methods. Methods from kinetic and meanfield theory will be extended in order to gain insight on properties and mechanisms of those learning approaches. The gained insight will used to propose novel, provable convergent and stable methods to solve learning problems. The derived meanfield and kinetic description will allow to perform theoretical studies to gain insight on their emergent behaviors. These include but are not limited to fast and stable training procedures, study of robustness with respect to uncertainty in parameters and the expressivity, as well as the data driven learning problems involving partial differential equations.

• DFG Programme: Priority Programmes
• Subproject of: SPP 2298: Theoretical Foundations of Deep Learning