Dean Hildebrand
ABSTRACT
MPI-IO is emerging as the standard mechanism for file I/O within HPC applications. While pNFS demonstrates high-performance I/O for bulk data transfers, its performance and scalability with MPI-IO is unproven. To attain success, the consistency semantics and interfaces of pNFS, POSIX, and MPI-IO must all be reconciled and efficiently translated. This paper investigates and discusses the challenges of using pNFS to support the consistency semantics of HPC applications.
- W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster, "The Globus Striped GridFTP Framework and Server," in Proceedings of Supercomputing '05, Seattle, WA, 2005. Google Scholar
Digital Library
- B. Welch, M. Unangst, Z. Abbasi, G. Gibson, B. Mueller, J. Small, J. Zelenka, and B. Zhou, "Scalable Performance of the Panasas Parallel File System," in Proceedings of the 6th USENIX Conference on File and Storage Technologies, San Jose, CA, 2008. Google ScholarDigital Library
- F. Schmuck and R. Haskin, "GPFS: A Shared-Disk File System for Large Computing Clusters," in Proceedings of the USENIX Conference on File and Storage Technologies, San Francisco, CA, 2002. Google ScholarDigital Library
- Cluster File Systems Inc., "Lustre: A Scalable, High-Performance File System," www.lustre.org, 2002.Google Scholar
- W. Gropp, S. Huss-Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir, MPI: The Complete Reference, volume 2--The MPI-2 Extensions. Cambridge, MA: MIT Press, 1998. Google ScholarDigital Library
- NCSA, "HDF5 ", hdf.ncsa.uiuc.edu/HDF5.Google Scholar
- J. Li, W. Liao, A. Choudhary, R. Ross, R. Thakur, W. Gropp, R. Latham, A. Siegel, B. Gallagher, and M. Zingale, "Parallel netCDF: A Scientific High-Performance I/O Interface," in Proceedings of Supercomputing '03, Phoenix, AZ, 2003. Google ScholarDigital Library
- R. Thakur, W. Gropp, and E. Lusk, "Data Sieving and Collective I/O in ROMIO," in Proceedings of the 7th Symposium on the Frontiers of Massively Parallel Computation, 1999. Google ScholarDigital Library
- S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M. Eisler, and D. Noveck, "NFS Version 4 Protocol Specification," RFC 3530, 2003.Google Scholar
- S. Shepler, M. Eisler, and D. Noveck, "NFSv4 Minor Version 1," Internet Draft, 2008.Google Scholar
- D. Hildebrand, P. Andrews, M. Eshel, R. Haskin, P. Kovatch, and J. White, "Deploying pNFS across the WAN: First Steps in HPC Grid Computing," in Proceedings of the 9th LCI International Conference on High-Performance Clustered Computing, Urbana, IL, 2008.Google Scholar
- D. Hildebrand and P. Honeyman, "Exporting Storage Systems in a Scalable Manner with pNFS," in Proceedings of the 22nd IEEE/13th NASA Goddard Conference on Mass Storage Systems and Technologies, Monterey, CA, 2005. Google ScholarDigital Library
- G. Grider, L. Ward, R. Ross, and G. Gibson, "A Business Case for Extensions to the POSIX I/O API for High End, Clustered, and Highly Concurrent Computing," www.opengroup.org/platform/hecewg, 2006.Google Scholar
- "MPI-Forum," www.mpi-forum.org/docs/_mpi-20-html/node215.htm.Google Scholar
- R. Thakur, W. Gropp, and E. Lusk, "An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces," in Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation, 1996. Google ScholarDigital Library
- R. Thakur, E. Lusk, and W. Gropp, "Users Guide for ROMIO: A High-Performance, Portable MPI-IO Implementation," Technical Memorandum ANL/MCS-TM-234, Mathematics and Computer Science Division, Argonne National Laboratory, Revised May 2004.Google Scholar
- M. Oberg, H. M. Tufo, and M. Woitaszek, "Exploration of Parallel Storage Architectures for a Blue Gene/L on the TeraGrid," in Proceedings of the 9th LCI International Conference on High-Performance Clustered Computing, Urbana, IL, 2008.Google Scholar
- D. Hildebrand, L. Ward, and P. Honeyman, "Large Files, Small Writes, and pNFS," in Proceedings of the 20th ACM International Conference on Supercomputing, Cairns, Australia, 2006. Google ScholarDigital Library
- Sun Microsystems Inc., "Lustre File System," White-paper, 2007.Google Scholar
- J. P. Prost, R. Treumann, R. Hedges, B. Jia, and A. E. Koniges, "MPI-IO/GPFS, an Optimized Implementation of MPI-IO on top of GPFS," in Proceedings of Supercomputing '01, Denver, CO, 2001. Google ScholarDigital Library
- P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur, "PVFS: A Parallel File System for Linux Clusters," in Proceedings of the 4th Annual Linux Showcase and Conference, Atlanta, GA, 2000. Google ScholarDigital Library
- I. Chavis, D. Coutts, J. Huie, S. Liu, S. Qualters, B. Demkowicz, and D. L Turkenkopf, "A Guide to the IBM Clustered Network File System," IBM Redbooks, 2008.Google Scholar
- IBM Corp., "IBM Storage Optimization and Integration Services-scale out file services," datasheet, 2007.Google Scholar
- M. Eisler, P. Corbett, M. Kazar, D. Nydick, and C. Wagner, "Data ONTAP GX: A Scalable Storage Cluster," in Proceedings of the 5th USENIX conference on File and Storage Technologies, San Jose, CA, 2007. Google ScholarDigital Library
Index Terms
- pNFS, POSIX, and MPI-IO: a tale of three semantics
Recommendations
Large files, small writes, and pNFS
ICS '06: Proceedings of the 20th annual international conference on SupercomputingWorkload characterization studies highlight the prevalence of small and sequential data requests in scientific applications. Parallel file systems excel at large data transfers but sometimes at the expense of small I/O performance. pNFS is an NFSv4.1 ...
pNFS/PVFS2 over InfiniBand: early experiences
PDSW '07: Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07The computing power of clusters has been rapidly growing up towards petascale capability, which requires petascale I/O systems to provide data in a sustained high-throughput manner. Network File System (NFS), a ubiquitous standard used in most existing ...
Direct-pNFS: scalable, transparent, and versatile access to parallel file systems
HPDC '07: Proceedings of the 16th international symposium on High performance distributed computingGrid computations require global access to massive data stores. To meet this need, the GridNFS project aims to provide scalable, high-performance, transparent, and secure wide-area data management as well as a scalable and agile name space.
While ...
Comments