Spiking Neural Networks: Theoretical Foundations, Trustworthiness, and Energy-Efficiency

• Applicants: Professor Dr.-Ing. Holger Boche; Professorin Dr. Gitta Kutyniok
• Subject Area: Mathematics ; Communication Technology and Networks, High-Frequency Technology and Photonic Systems, Signal Processing and Machine Learning for Information Technology
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 543965165

While deep learning has shown remarkable success, it is crucial to acknowledge and address its notable limitations and drawbacks. At present, deep learning methods face inherent limitations related to trustworthiness, with the term "trustworthiness" encompassing aspects such as privacy, security, resilience, reliability, and accountability. It was recently shown that the success of algorithmic computation in deep learning may even depend on the applied computing platform. In particular, the ubiquitous digital hardware, where deep learning models are typically implemented on, has restrictions that may prevent trustworthy deep learning in specific circumstances. In addition to trustworthiness, another concern related to deep learning is the rapidly increasing need of computational power. As models become more complex and demand greater computational resources, there is a growing realization that the current technologies may not be sustainable in the long term due to their energy requirements. The central question emerges: Can we overcome these challenges to create trustworthy and energy-efficient Artificial Intelligence? One possible solution, without compromising the capabilities of deep learning, may be the deployment of spiking neural networks on neuromorphic hardware. This would represent a shift on the software side - from basic neural networks to more evolved spiking neural networks - and on the hardware side - from a classical digital computing model to a neuromorphic approach optimized to benefit the implementation of spiking neural networks. The goal of this proposal is to theoretically analyze spiking neural networks combined with neuromorphic hardware in the framework of trustworthy and energy-efficient computations. Our key objectives are to (1) investigate the expressive capabilities of spiking neural networks specifically through the Spike Response and Leaky-Integrate and Fire Model, (2) analyze theoretical aspects of the algorithms to effectively train spiking neural networks, derive generalization bounds and conduct comprehensive numerical experiments to validate the findings, (3) introduce metrics designed to measure energy-efficiency and perform a comparative analysis of spiking network implementation on both digital and neuromorphic hardware, and (4) study different computing models and corresponding hardware platforms for the implementation of trustworthy spiking neural networks.

• DFG Programme: Priority Programmes
• Subproject of: SPP 2298: Theoretical Foundations of Deep Learning