Distributed Monitoring in Large Scale Overlay Networks (OverlayMeter)
Final Report Abstract
Peer-to-peer systems, as well as centralized systems, aim at utilizing the available resources to the best to provide a high quality service. Here peer-to-peer systems have the potential to easily scale with the number of users, as every users brings new resources to the system. However, in a decentralized system it is challenging to maintain a desired quality level. This is due to the fact that participants have to make decisions based on their locally available information. To make the best decisions, a solid and extensive data basis is indispensable. For this purpose the literature on the field of peer-to-peer networks proposes monitoring the system, an approach that we follow in this work. Monitoring refers to the gathering and dissemination of system- and peer-specific data. In this project, we first deal with monitoring procedures in the system-specific context. Here, we focus on tree-based procedures, as we have seen the biggest potential for this class, and tackle existing problems in their robustness. We propose mechanisms for redundancy in a smart distribution function and the proposal of additional mechanisms that improve the perception of each participant in the monitoring structure. Thus, we present two methods that significantly increase the robustness, keep the precision on a consistently high level and have a good costbenefit ratio. In addition to the improvement of existing monitoring systems for system-specific data, we treat phenomena that lead to an impairment of the monitoring result. The influence of malicious participants in monitoring procedures is not adequately analyzed in the literature and existing solutions for the general handling of such participants are complex and could open doors for further attack vectors. In a comprehensive evaluation, we show that manipulation attacks can be limited through our proposed approach to a convex hull based on outlier detection and attacks on the monitoring structure can be reduced by verifying their origin. Overall, the proposed extension for tree-based monitoring systems operates passively and is therefore versatile. On the other hand, we are pursuing an undisclosed field in monitoring; the incomplete participation of users in monitoring solutions. We identify the negative impact that such a behavior has on the accuracy of the monitoring and present a solution solving this problem. As a solution, we propose a generic middleware that reuses existing monitoring procedures. Here, we rely on an organization of the active participants in the middleware to measure the passive participants by probing them and feeding captured information into the monitoring solution. The evaluation shows a high degree of precision, which among others depends on the precision of the probing methods. As a second point, we address the peer-specific data gathering and the capacity-based peer search. The goal is to create an efficient indexing structure that efficiently stores highly dynamic peers’ heterogeneous capacities in a distributed manner. We aim on an accurate peer search mechanism that obtains peers fulfilling set requirements. Such systems are motivated with the delegation of tasks and the realization of distributed computing in a decentralized context. We propose two solutions that efficiently index highly dynamic peer capacities and distribute the load fairly among all participants. While our first solution globally sets the number of capacities to a constant, our second solution uses a dynamic approach. The proposed search processes perform quickly and precisely, so that they meet the demands. As third point, we focus on a barrier which arises when using peer-to-peer software. It is the mandatory installation and set up of third-party software prior to running it. We present two prototypes using the WebRTC standard to establish direct connections between browser instances, thus offering the opportunity of using peer-to-peer techniques in the context of the web. We combine the presented methods to the so-called OverlayMeter, which provides a basis for extensive monitoring. This basis includes accurate and cost-effective system- and peer-specific data monitoring along with capacity-based peer search. The methods of the OverlayMeter can finally be utilized in various peer-to-peer applications to reliably and precisely monitor the network.
Publications
- “Chunked-Swarm: Divide and Conquer for Real-time Bounds in Video Streaming“. In: Proceedings of the 15 th IEEE International Conference on Next Generation Wired/Wireless Advanced Networks and Systems (New2An). 2015
Christopher Probst, Andreas Disterhöft and Kalman Graffi
(See online at https://doi.org/10.1007/978-3-319-23126-6_18) - “Convex Hull Watchdog: Mitigation of Malicious Nodes in Tree-Based P2P Monitoring Systems“. In: Proceedings of the 41 st IEEE International Conference on Local Computer Networks (LCN) 2016
Andreas Disterhöft and Kalman Graffi
(See online at https://doi.org/10.1109/LCN.2016.16) - “CapSearch: Capacity-Based Search in Highly Dynamic Peer-to-Peer Networks”. In: Proceedings of the 31 st IEEE International Conference on Advanced Information Networking and Applications (AINA). 2017
Andreas Disterhöft and Kalman Graffi
(See online at https://doi.org/10.1109/AINA.2017.129) - “Minicamp: Prototype for Partial Participation in Structured Peer-to-Peer Monitoring Protocols“. In: Proceedings of the 42 nd IEEE International Conference on Local Computer Networks (LCN) 2017
Andreas Disterhöft and Kalman Graffi
(See online at https://doi.org/10.1109/LCN.2017.58) - “PacketSkip: Skip Graph for Multi-dimensional Search in Structured Peer-to-Peer Systems“. In: Proceedings of the 11 th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO). 2017
Andreas Disterhöft, Andreas Funke and Kalman Graffi
(See online at https://doi.org/10.1109/SASO.2017.11) - “SkyEye: A tree-based peer-to-peer monitoring approach“. In: Elsevier Pervasive and Mobile Computing, Volume 40. 2017
Kalman Graffi and Andreas Disterhöft
(See online at https://doi.org/10.1016/j.pmcj.2017.07.003) - “Minicamp: Middleware for Incomplete Participation in Structured Peer-to-Peer Monitoring Protocols“. In: Proceedings of the 32 nd IEEE Inter-national Conference on Advanced Information Networking and Applications (AINA). 2018
Andreas Disterhöft and Kalman Graffi
(See online at https://doi.org/10.1109/AINA.2018.00150) - “Mr.Tree: Multiple Realities in Tree-based Monitoring Overlays for Peer-to-Peer Networks“. In: Proceedings of the IEEE International Conference on Computing, Networking and Communications (ICNC) 2018
Andreas Disterhöft, Phillip Sandkühler, Andre Ippisch and Kalman Graffi
(See online at https://doi.org/10.1109/ICCNC.2018.8390361)
