2006 | OriginalPaper | Chapter
A New I/O Architecture for Improving the Performance in Large Scale Clusters
Authors : L. M. Sánchez García, Florin D. Isaila, Félix García Carballeira, Jesús Carretero Pérez, Rolf Rabenseifner, Panagiotis Adamidis
Published in: Computational Science and Its Applications - ICCSA 2006
Publisher: Springer Berlin Heidelberg
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
The technology advances made in supercomputers and high performance computing clusters over the past few years have been tremendous. Clusters are the most common solution for high performance computing at the present time. In this kind of systems, an important subject is the parallel I/O subsystem design. Parallel file systems (GPFS, PVFS, Lustre, etc) have been the solution used to obtain high performance I/O. Parallel file systems increase performance by distributing data file across several I/O nodes. However, cluster’s size is increasing continuously, specially for compute nodes, becoming the I/O nodes in a possible bottleneck of the system.
In this paper, we propose a new architecture that solves the problem pointed out before: new hierarchical I/O architecture based on parallel I/O proxies. Those I/O proxies execute on the compute nodes offering an intermediate parallel file system between the applications and the storage system of the cluster. That architecture reduces the load on the I/O nodes increasing the global performance. This paper shows the design of the proposed solution and a preliminary evaluation, using a cluster located in the Stuttgart HLRS center.