Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2021 | OriginalPaper | Chapter

Verification, Validation and Uncertainty Quantification of Large-Scale Applications with QCG-PilotJob

Authors: Bartosz Bosak, Tomasz Piontek, Paul Karlshoefer, Erwan Raffin, Jalal Lakhlili, Piotr Kopta

Published in: Computational Science – ICCS 2021

Publisher: Springer International Publishing

share
SHARE

Abstract

Efficient execution of large-scale and extremely demanding computational scenarios is a challenge for both the infrastructure providers and end-users, usually scientists, that need to develop highly scalable computational codes. Nevertheless, at this time, on the eve of exa-scale supercomputers, the particular role has to be given also to the intermediate software that can help in the preparation of applications so they can be efficiently executed on the emerging HPC systems. The efficiency and scalability of such software can be seen as priorities, however, these are not the only elements that should be addressed. Equally important is to offer software that is elastic, portable between platforms of different sizes, and easy to use. Trying to fulfill all the above needs we present QCG-PilotJob, a tool designed to enable flexible execution of numerous potentially dynamic and interdependent computing tasks in a single allocation on a computing cluster. QCG-PilotJob is built on many years of collaboration with computational scientists representing various domains and it responses to the practical requirements of real scientific use-cases. In this paper, we focus on the recent integration of QCG-PilotJob with the EasyVVUQ library and its successful use for Uncertainty Quantification workflows of several complex multiscale applications being developed within the VECMA project. However, we believe that with a well-thought-out design that allows for fully user-space execution and straightforward installation, QCG-PilotJob may be easily exploited in many other application scenarios, even by inexperienced users.
Literature
6.
go back to reference Merzky, A., Turilli, M., Titov, M., Al-Saadi, A., Jha, S.: Design and performance characterization of RADICAL-PILOT on leadership-class platforms (2021) Merzky, A., Turilli, M., Titov, M., Al-Saadi, A., Jha, S.: Design and performance characterization of RADICAL-PILOT on leadership-class platforms (2021)
7.
go back to reference Piontek, T., et al.: Development of science gateways using QCG – lessons learned from the deployment on large scale distributed and HPC infrastructures. J. Grid Comput. 14, 559–573 (2016) CrossRef Piontek, T., et al.: Development of science gateways using QCG – lessons learned from the deployment on large scale distributed and HPC infrastructures. J. Grid Comput. 14, 559–573 (2016) CrossRef
8.
go back to reference Richardson, R., Wright, D., Edeling, W., Jancauskas, V., Lakhlili, J., Coveney, P.: EasyVVUQ: a library for verification, validation and uncertainty quantification in high performance computing. J. Open Res. Softw. 8, 1–8 (2020) CrossRef Richardson, R., Wright, D., Edeling, W., Jancauskas, V., Lakhlili, J., Coveney, P.: EasyVVUQ: a library for verification, validation and uncertainty quantification in high performance computing. J. Open Res. Softw. 8, 1–8 (2020) CrossRef
9.
10.
go back to reference Wright, D.W., et al.: Building confidence in simulation: applications of EasyVVUQ. Adv. Theory Simul. 3(8), 1900246 (2020) CrossRef Wright, D.W., et al.: Building confidence in simulation: applications of EasyVVUQ. Adv. Theory Simul. 3(8), 1900246 (2020) CrossRef
Metadata
Title
Verification, Validation and Uncertainty Quantification of Large-Scale Applications with QCG-PilotJob
Authors
Bartosz Bosak
Tomasz Piontek
Paul Karlshoefer
Erwan Raffin
Jalal Lakhlili
Piotr Kopta
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-77977-1_39