ExtraNoise - Performance analysis of HPC applications in noisy environments

• Applicants: Dr.-Ing. Bernd Mohr; Professor Dr. Felix Wolf
• Subject Area: Data Management, Data-Intensive Systems, Computer Science Methods in Business Informatics
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 449683531

High-performance computing is a key technology of the 21st century. Numerous application examples, ranging from the improved understanding of matter to the discovery of new materials and from the study of biological processes to artificial intelligence, give evidence of its tremendous potential. However, exploiting the full power of HPC systems has always been hard and is becoming even harder as the complexity and size of systems and applications continue to grow. On the flipside, the savings potential in terms of energy and CPU hours that application optimization can achieve is enormous. Key to understanding and ultimately improving the performance of HPC applications is performance measurement. Unfortunately, many HPC systems expose their jobs to substantial amounts of interference (aka noise), leading to significant run-to-run variation. This makes performance measurements generally irreproducible, heavily complicating performance analysis and modeling. On noisy systems, performance analysts usually have to repeat performance measurements several times and then apply statistics to capture trends. First, this is expensive and, second, extracting trends from a limited series of experiments is far from trivial, as the noise can follow quite irregular patterns. The goal of this project is, on the one hand, to develop methods that make performance analysis more noise resilient and, on the other hand, to better understand how applications respond to noise in general and which design choices increase or lower their active and passive interference potential.

• DFG Programme: Research Grants
• International Connection: Russia
• Partner Organisation: Russian Foundation for Basic Research
• Cooperation Partner: Dr. Dmitry Nikitenko