Process-Oriented Performance Engineering Service Infrastructure for Scientific Software at German HPC Centers

• Applicants: Professor Dr. Matthias S. Müller; Professor Dr. Wolfgang E. Nagel; Professor Dr. Gerhard Wellein
• Subject Area: Data Management, Data-Intensive Systems, Computer Science Methods in Business Informatics
• Term: from 2016 to 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 320899119

The ProPE project will deploy a prototype HPC user support infrastructure as a distributed cross-site collaborative effort of several tier-2/3 centers with complementing HPC expertise. Within ProPE code optimizing and parallelization of scientific software is seen as a structured, well-defined process with sustainable outcome.The central component of ProPE is the improvement, process-based implementation, and dissemination of a structured performance engineering (PE) process. This PE process defines and drives code optimization and parallelization as a target-oriented, structured process. Application hot spots are identified first and then optimized/parallelized in an iterative cycle: Starting with an analysis of the algorithm, the code, and the target hardware a hypothesis of the performance-limiting factors is proposed based on performance patterns and models. Performance measurements validate or guide the iterative adaption of the hypothesis. After validation of the hardware bottleneck, appropriate code changes are deployed and the PE cycle restarts. The level of detail of the PE process can be adapted to the complexity of the underlying problem and the experience of the HPC analyst. Currently this process is applied by experts and at the prototype level. ProPE will formalize and document the PE process and apply it to various scenarios (single core/node optimization, distributed parallelization, IO-intensive problems). Different abstraction levels of the PE process will be implemented and disseminated to HPC analysts and application developers via user support projects, teaching activities, and web documentation. The integration of the PE process into modern IT infrastructure across several centers with different HPC support expertise will be the second project focus. All components of the PE process will be coordinated and standardized across the partnering sites. This way the complete HPC expertise within ProPE can be offered as coherent service on a nationwide scale. Ongoing support projects can be transferred easily between participating centers. In order to identify low-performing applications, characterize application loads, and quantify benefits of the PE activities at a system level, ProPE will employ a system monitoring infrastructure for HPC clusters. This tool will be tailored to the requirements of the PE process and designed for easy deployment and usage at tier-2/3 centers.The associated ProPE partners will ensure the embedding into the German HPC infrastructure and provide basic PE expertise in terms of algorithmic choices, perfectly complementing the code optimization and parallelization efforts of ProPE.

• DFG Programme: Research Grants
• Cooperation Partner: Professor Dr. Hans-Joachim Bungartz