Compiler-based correctness checks for SPMD applications

The problems that are being worked on in scientific computing and high performance computing today are highly complex and require immense computing resources. Due to the scale of applications such as, for example, climate simulations they need to be run on clusters that consist of many networked computing nodes. To utilize these resources, an application is required to be implemented using parallel distributed programming models. The standard method used for this in high performance computing is called Single Program, Multiple Data (SPMD). An application spawns multiple processes that work on the same problem and communicate with each other via message passing. While this allows for a high scalability, the SPMD model is not easy to program and many new types of programming errors can arise.

In this project, we are developing a compiler-based tool called SPMDClang, which supports the developers of SPMD applications with compile time correctness checks. It is based on the Clang frontend of the LLVM compiler toolchain. The goal is to provide compiler warnings and errors about the correctness of the communication schemes of SPMD codes during compilation. This requires the static analysis of the structure of the message passing operations in a program and newly developed algorithms to detect potential problems with the communication patterns. Due to the fact that some important program parameters such as the number of processes that will be used at runtime are not clear at compile time, a symbolic execution approach is required to simulate the potential runtime behavior of the SPMD application. For this an approach of using colored Petri nets to simulate the runtime behavior of the analyzed program is being developed. With this project, we aim to ascertain to what degree it is possible to detect SPMD-related error classes with a static compile time approach since most existing work in this area is based on runtime correctness checks. Additionally, a relevant question is how strongly the symbolic execution analysis will impact the compile time of an application. The goal is to provide a user-friendly tool that makes the development of SPMD applications easier on developers. We have the opportunity to evaluate these aspects by incorporating the developed tool in our courses on parallel programming where the development of SPMD applications is taught in the exercises.