Skip to main content

2018 | OriginalPaper | Buchkapitel

Efficient Implementation of Data Objects in the OSD+-Based Fusion Parallel File System

verfasst von : Juan Piernas, Pilar González-Férez

Erschienen in: Euro-Par 2017: Parallel Processing Workshops

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

OSD+s are enhanced object-based storage devices (OSDs) able to deal with both data and metadata operations via data and directory objects, respectively. So far, we have focused on designing and implementing efficient directory objects in OSD+s. This paper, however, presents our work on also supporting data objects, and describes how the coexistence of both kinds of objects in OSD+s is profited to efficiently implement data objects and to speed up some common file operations. We compare our OSD+-based Fusion Parallel File System (FPFS) with Lustre and OrangeFS. Results show that FPFS provides a performance up to \(37{\times }\) better than Lustre, and up to \(95{\times }\) better than OrangeFS, for metadata workloads. FPFS also provides 34% more bandwidth than OrangeFS for data workloads, and competes with Lustre for data writes. Results also show serious scalability problems in Lustre and OrangeFS.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Avilés-González, A., Piernas, J., González-Férez, P.: Scalable metadata management through OSD+ devices. Int. J. Parallel Program. 42(1), 4–29 (2014)CrossRef Avilés-González, A., Piernas, J., González-Férez, P.: Scalable metadata management through OSD+ devices. Int. J. Parallel Program. 42(1), 4–29 (2014)CrossRef
2.
Zurück zum Zitat Avilés-González, A., Piernas, J., González-Férez, P.: Batching operations to improve the performance of a distributed metadata service. J. Supercomput. 72(2), 654–687 (2016)CrossRef Avilés-González, A., Piernas, J., González-Férez, P.: Batching operations to improve the performance of a distributed metadata service. J. Supercomput. 72(2), 654–687 (2016)CrossRef
3.
Zurück zum Zitat Bent, J., Gibson, G., Grider, G., McClelland, B., Nowoczynski, P., Nunez, J., Polte, M., Wingate, M.: PLFS: a checkpoint filesystem for parallel applications. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC 2009), pp. 1–12 (2009) Bent, J., Gibson, G., Grider, G., McClelland, B., Nowoczynski, P., Nunez, J., Polte, M., Wingate, M.: PLFS: a checkpoint filesystem for parallel applications. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC 2009), pp. 1–12 (2009)
4.
Zurück zum Zitat Brandt, S.A., Miller, E.L., Long, D.D.E., Xue., L.: Efficient metadata management in large distributed storage systems. In: Proceedings of the 20th IEEE Conference on Mass Storage Systems and Technologies (MSST 2003), pp. 290–298 (2003) Brandt, S.A., Miller, E.L., Long, D.D.E., Xue., L.: Efficient metadata management in large distributed storage systems. In: Proceedings of the 20th IEEE Conference on Mass Storage Systems and Technologies (MSST 2003), pp. 290–298 (2003)
8.
Zurück zum Zitat Patil, S., Ren, K., Gibson, G.: A case for scaling HPC metadata performance through de-specialization. In: Proceedings of 7th Petascale Data Storage Workshop Supercomputing (PDSW 2012), pp. 1–6, November 2012 Patil, S., Ren, K., Gibson, G.: A case for scaling HPC metadata performance through de-specialization. In: Proceedings of 7th Petascale Data Storage Workshop Supercomputing (PDSW 2012), pp. 1–6, November 2012
9.
Zurück zum Zitat Skeen, D., Stonebraker, M.: A formal model of crash recovery in a distributed system. IEEE Trans. Softw. Eng. 9(3), 219–228 (1983)CrossRef Skeen, D., Stonebraker, M.: A formal model of crash recovery in a distributed system. IEEE Trans. Softw. Eng. 9(3), 219–228 (1983)CrossRef
11.
Zurück zum Zitat Weil, S.A., Brandt, S.A., Miller, E.L., Long, D.D.E., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2006), pp. 307–320 (2006) Weil, S.A., Brandt, S.A., Miller, E.L., Long, D.D.E., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2006), pp. 307–320 (2006)
Metadaten
Titel
Efficient Implementation of Data Objects in the OSD+-Based Fusion Parallel File System
verfasst von
Juan Piernas
Pilar González-Férez
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-75178-8_59

Neuer Inhalt