Advertisement
Published in:
2018 | OriginalPaper | Chapter
Accelerated Purge Processes of Parallel File System on HPC by Using MPI Programming
Authors : Min-Woo Kwon, JunWeon Yoon, TaeYoung Hong, ChanYeol Park
Published in: Advances in Computer Science and Ubiquitous Computing
Publisher: Springer Singapore
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
Abstract
HPC system usually uses a shared filesystem like Lustre as temporary file storage like scratch directory. It needs automated purge process to remove unused files for maintaining optimal performance of the shared filesystem. However, the purge process in large capacity file system takes much time to search and remove target files. In this paper, accelerated purge processes using MPI are proposed. First, master/slave parallel purge (MSPP) process is the method that master node distributes purge tasks among slave nodes. Second, evenly distributed purge (EDPP) process is the method that all node are involved in purge process that improves load balancing. Experimental results show that the purging time of proposed EDPP method for 1958 GB scratch data has been reduced by 4.04 and 2.47 times, respectively, when it is compared with the results of single node purge (SNP) and MSPP methods.