FogStore – A Data Management Platform for Geo-Distributed Fog Environments
Final Report Abstract
The “FogStore” project aimed to develop a data management system for fog computing, specifically targeting emerging application domains such as the Internet of Things (IoT), autonomous driving, and future mobile networks (5G, 6G). The project focused on addressing the challenges of data distribution, replica selection, and predictive replica placement in geo-distributed fog environments. The project followed a work plan that involved the development of an open-source software prototype called FReD (Fog Replicated Data) as the basis for the project. This prototype implemented key abstractions such as replica nodes, keygroups, and trigger nodes, which proved effective for flexible and efficient data replication in geo-distributed environments. However, the project encountered complexities in researching alternative consensus mechanisms for nodes and had to postpone this aspect to the second year. The project team realized that the trigger node abstraction alone was insufficient to support application developers in creating fog services. As a solution, they developed a lightweight serverless platform called tinyFaaS for managing compute services at the edge. Additionally, they explored predictive replica placement and leveraged machine learning mechanisms, specifically location prediction with Markov models, to anticipate future application requests and optimize replica placement. The project also focused on developing a benchmarking framework for evaluating fog data management systems’ performance in geo-distributed and coordinated execution of parallel workloads. Throughout the project, there were shifts in research attention due to the evolving landscape of mobile networks. Specifically, the anticipation of in-network computing capabilities did not materialize as 5G matured. These were shifted instead to 6G, which also brought along a focus on non-terrestrial low-Earth orbit (LEO) networks. The team extended their research to compute service and data management on LEO satellite networks, leading to fruitful results in terms of publications and the development of a virtual fog testbed tool called Celestial. The project’s main results include the development of FReD, an open-source data management system for the fog with application-controlled replica placement, which provides flexible and efficient replication options based on keygroups and replica nodes. The system offers client-centric data consistency guarantees and supports complex applications through trigger nodes. The project also introduced tinyFaaS, a lightweight serverless platform for the edge that enables elastic scaling of compute services with minimal resource usage. The team explored distributed coordination strategies based on layered coordination systems and investigated predictive replica placement techniques using machine learning models. Additionally, the team made significant contributions to the advancement of data and compute service management in LEO satellite networks with in-network computing capabilities. The project’s findings contribute to the advancement of fog computing research and offer insights into addressing the challenges of data management, replica placement, and coordination in geo-distributed fog environments. The results have potential applications in various domains and lay the groundwork for future investigations in fog computing and mobile network technologies.
Publications
-
tinyFaaS: A Lightweight FaaS Platform for Edge Environments. 2020 IEEE International Conference on Fog Computing (ICFC), 17-24. IEEE.
Pfandzelter, Tobias & Bermbach, David
-
Edge (of the Earth) Replication: Optimizing Content Delivery in Large LEO Satellite Communication Networks. 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid), 565-575. IEEE.
Pfandzelter, Tobias & Bermbach, David
-
From zero to fog: Efficient engineering of fog‐based Internet of Things applications. Software: Practice and Experience, 51(8), 1798-1821.
Pfandzelter, Tobias; Hasenburg, Jonathan & Bermbach, David
-
Predictive replica placement for mobile users in distributed fog data stores with client-side markov models. Proceedings of the 14th IEEE/ACM International Conference on Utility and Cloud Computing Companion, 1-8. ACM.
Bellmann, Malte; Pfandzelter, Tobias & Bermbach, David
-
Towards a Computing Platform for the LEO Edge. Proceedings of the 4th International Workshop on Edge Systems, Analytics and Networking, 43-48. ACM.
Pfandzelter, Tobias; Hasenburg, Jonathan & Bermbach, David
-
Celestial. Proceedings of the 23rd ACM/IFIP International Middleware Conference, 69-81. ACM.
Pfandzelter, Tobias & Bermbach, David
-
QoS-Aware Resource Placement for LEO Satellite Edge Computing. 2022 IEEE 6th International Conference on Fog and Edge Computing (ICFEC), 66-72. IEEE.
Pfandzelter, Tobias & Bermbach, David
-
Streaming vs. Functions: A Cost Perspective on Cloud Event Processing. 2022 IEEE International Conference on Cloud Engineering (IC2E), 67–78.
Pfandzelter, Tobias; Henning, Soren; Schirmer, Trever; Hasselbring, Wilhelm & Bermbach, David
-
Managing data replication and distribution in the fog with FReD. Software: Practice and Experience, 53(10), 1958-1981.
Pfandzelter, Tobias; Japke, Nils; Schirmer, Trever; Hasenburg, Jonathan & Bermbach, David
-
Towards a Benchmark for Fog Data Processing. 2023 IEEE International Conference on Cloud Engineering (IC2E), 92-98. IEEE.
Pfandzelter, Tobias & Bermbach, David
