General resource manager for computationally demanding scientific software (MARE)

Today’s supercomputers power scientific calculations in very different areas, ranging from nanotechnology to climate studies and astronomy. Research groups in nascent areas of science and engineering develop their own scientific software, since it cannot be obtained otherwise. These groups are typically not funded to spend time on performance optimization and software design, but to address their original research questions. In more established fields, several accepted research applications emerge as a standard and get adopted by research groups that are users, with little software development experience. The variability of applications yields a hard to optimize and sub-optimal supercomputer resource usage. Large computing times or large memory requirements often limit the scope of the research exploration and a more optimal use can reduce overall compute time or increase the scientific scope of questions being asked. This work introduces a resource manager that requires minimal intrusion and only superficial understanding of the scientific code, but still schedules the execution of the calculations to optimally utilize memory, central processing unit (CPU) or both. This is exemplified with Quantum Espresso and recursive open boundary and interfaces (ROBIN) calculations on regional and national computer infrastructures. The resource manager presented here reduces compute times by 13–30% in those two scientifically relevant computational codes.