RoCELeaN: Robust and Communication-Efficient Learning over Networks

• Applicant: Professor Dr.-Ing. Abdelhak Zoubir
• Subject Area: Communication Technology and Networks, High-Frequency Technology and Photonic Systems, Signal Processing and Machine Learning for Information Technology
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 564846918

In distributed learning, a number of nodes that form a network aim to collaboratively infer a set of parameters based on their local data as well as on information exchanged within the neighborhood of the nodes. A variety of problems, ranging from distributed regression to distributed deep learning can be solved within this framework. In many networks, the sensors are spatially dispersed and, hence, some of the sensors may observe data that is corrupted by gross outliers. These outliers will, due to the information exchange in the network, cause the overall algorithm to break down after some iterations. This calls for robust methods that are insensitive to gross outliers and other model deviations. Here, the spirit is to sacrifice some efficiency under ideal conditions in order to tolerate deviations from the ideal case. Moreover, the size of current models increases continuously and hence, transmitting an update comes with a high communication burden. Therefore, controlling the communication load while retaining a fast convergence speed is crucial, which could, for example, be achieved by only communicating sparse messages. In the RoCELeaN project, we plan to jointly solve the three aforementioned areas: distributed learning, statistical robustness, and communication efficiency by considering sparse messaging. This problem is of particular importance as many modern applications, for example, smart cities or decentralized energy systems, tend to use an ever growing number of sensors. To the best of our knowledge, existing, but limited work only deals with robust or communication-efficient distributed learning. The most important open questions in this project are summarized below: 1. How can messages generated and aggregated in a way that is robust against different scenarios, including outliers or malicious nodes? 2. How can the communication bandwidth be reduced by only considering sparse messages while not losing much convergence speed? 3. How can sparse messages be generated and aggregated in a way that is insensitive to different scenarios, including gross outliers or malicious nodes? A successful completion of the project would result in a coherent framework coming with practical algorithms as well as theoretical results, such as performance guarantees and bounds. The Signal Processing Group is internationally recognized for its work on robust statistics as well as distributed and sparse signal processing. Therefore, we see ourselves in a uniquely favorable position for a successful completion of the proposed project, which brings together these core areas of expertise.

• DFG Programme: Research Grants