Servicenavigation

Project Details

Back

Projekt Print View

Coded Distributed Computation with Application to Machine Learning

Subject Area Communication Technology and Networks, High-Frequency Technology and Photonic Systems, Signal Processing and Machine Learning for Information Technology
Term from 2021 to 2025
Project identifier Deutsche Forschungsgemeinschaft (DFG) - Project number 470027923
 
Final Report Year 2025

Final Report Abstract

In this project, our goal was to utilize channel coding tools to enhance the speed, security, and privacy of distributed computing and federated learning. In order to process a large amount of data for training good machine learning models (like deep neural networks, large language models, etc.), distributing computations to capable servers becomes necessary. However, distributed computing introduces issues such as stragglers (nodes with lower computation and communication capabilities) and privacy and security concerns (due to the usage of external computing servers). Federated learning is a machine learning paradigm where the data are a property of decentralized users, hence imposing similar issues as distributed computing. Our goals in this project were as follows. i) constructing coding schemes that are fast, private, and secure for distributed computing, ii) deriving fundamental bounds and limitations for coding schemes for distributed computing, and iii) constructing coding schemes for federated learning to account for its straggler-tolerance, privacy, and security concerns. To achieve these goals, we utilized techniques from algebraic channel codes and from codes on graphs (also known as modern channel coding) to provide resiliency over straggling computation nodes and to ensure privacy and security. Moreover, we adopted well-known techniques related to channel coding, such as group testing, to achieve our goals. Among our results are the first private rateless matrix multiplication scheme, where we use Fountain codes in order to construct a rateless scheme to speed up matrix multiplication and use Lagrange coded computing in order to preserve the privacy of the matrices being multiplied. Moreover, we upgraded the scheme by using verification techniques in order to ensure resiliency against malicious attackers. We focused on the problem of privacy-preserving techniques over data stored in sparse matrices, since existing techniques did not account for this type of matrix. In this project, we also focused on speeding up federated learning (also known as coded federated learning) by replicating the data in a private way across different clients. We were the first to investigate the use of computationally private techniques based on codebased cryptography to preserve privacy in coded federated learning. An important aspect in federated learning is also the combination of privacy-preserving techniques and security-guaranteeing ones. To ensure that, we adopted techniques from group testing in order to allow a private identification of malicious clients in federated learning, using secure aggregation as a privacy-preserving technique.

Publications

 
 

Additional Information

© 2026 DFG Disclaimer / Copyright / Privacy Policy

Textvergrößerung und Kontrastanpassung