Skip to main content

2020 | OriginalPaper | Buchkapitel

Data Exchange Engine for Parallel Computing and Its Application to 3D Chromosome Modelling

verfasst von : Xiaoling Zhang, Yao Lu, Junfeng Wu, Yongrui Zhang

Erschienen in: Data Science

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Data Exchange Engine for Parallel Computing (abbreviated as DEEPC) is a universal parallel programming interface for scientific computing environments such as MATLAB, Octave, R and Python. It is a software developed by us to support Bulk Synchronous Parallel (BSP) computing for these mainstream script-driven scientific computing environments. BSP is one of the most dominant parallel program models, and it affects the design of parallel algorithms profoundly. However, most of these scientific computing environments have been lack of the software support of BSP for a long time until the birth of DEEPC. The main features of our DEEPC is its ease of use and high performance, especially that without much modification to the sequential-computing programs, one can combine these programs to a high performance parallel program with a short script. To demonstrate these features, we put DEEPC in use to a MATLAB program for the 3D modelling of chromosomes. It has been observed that DEEPC performs very well even without much modification to the corresponding program for sequential computing.
CCS Concepts. Computing methodologies → Parallel programming

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bisseling, R.H.: Parallel Scientific Computation: A Structured Approach Using BSP and MPI (2004) Bisseling, R.H.: Parallel Scientific Computation: A Structured Approach Using BSP and MPI (2004)
2.
Zurück zum Zitat Bonorden, O., Huppelshauser, N., Juurlink, B., Rieping, I.: The Paderborn University BSP (PUB) library on the CRAY T3E (2000) Bonorden, O., Huppelshauser, N., Juurlink, B., Rieping, I.: The Paderborn University BSP (PUB) library on the CRAY T3E (2000)
3.
Zurück zum Zitat Bonorden, O., Juurlink, B., Von Otte, I., Rieping, I.: The Paderborn University BSP (Pub) library-design, implementation and performance. In: Parallel Processing, 1999, 13th International and 10th Symposium on Parallel and Distributed Processing, 1999 IPPS/SPDP. Proceedings, pp. 99–104. IEEE (1999) Bonorden, O., Juurlink, B., Von Otte, I., Rieping, I.: The Paderborn University BSP (Pub) library-design, implementation and performance. In: Parallel Processing, 1999, 13th International and 10th Symposium on Parallel and Distributed Processing, 1999 IPPS/SPDP. Proceedings, pp. 99–104. IEEE (1999)
5.
Zurück zum Zitat Ching, A.: Scaling apache giraph to a trillion edges. Facebook Engineering blog (2013) Ching, A.: Scaling apache giraph to a trillion edges. Facebook Engineering blog (2013)
6.
Zurück zum Zitat Ching, A., Kunz, C.: Apache giraph (2013) Ching, A., Kunz, C.: Apache giraph (2013)
7.
Zurück zum Zitat Loulergue, F., Gava, F., Billiet, D.: Bulk synchronous parallel ML: modular implementation and performance prediction. In: Sunderam, V.S., van Albada, G.D., Sloot, P.M.A., Dongarra, J.J. (eds.) ICCS 2005. LNCS, vol. 3515, pp. 1046–1054. Springer, Heidelberg (2005). https://doi.org/10.1007/11428848_132CrossRef Loulergue, F., Gava, F., Billiet, D.: Bulk synchronous parallel ML: modular implementation and performance prediction. In: Sunderam, V.S., van Albada, G.D., Sloot, P.M.A., Dongarra, J.J. (eds.) ICCS 2005. LNCS, vol. 3515, pp. 1046–1054. Springer, Heidelberg (2005). https://​doi.​org/​10.​1007/​11428848_​132CrossRef
8.
Zurück zum Zitat Hill, J., Donaldson, S.R., Skillicorn, D.B.: Portability of performance with the BSPlib communications library. In: Third Working Conference on Massively Parallel Programming Models, Proceedings, pp. 33–42. IEEE (1997) Hill, J., Donaldson, S.R., Skillicorn, D.B.: Portability of performance with the BSPlib communications library. In: Third Working Conference on Massively Parallel Programming Models, Proceedings, pp. 33–42. IEEE (1997)
9.
Zurück zum Zitat Hill, J.M.D., Jarvis, S.A., Siniolakis, C., Vasilev, V.P.: Analysing an SQL application with a BSPlib call-graph profiling tool. In: Pritchard, D., Reeve, J. (eds.) Euro-Par 1998. LNCS, vol. 1470, pp. 157–164. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0057848CrossRef Hill, J.M.D., Jarvis, S.A., Siniolakis, C., Vasilev, V.P.: Analysing an SQL application with a BSPlib call-graph profiling tool. In: Pritchard, D., Reeve, J. (eds.) Euro-Par 1998. LNCS, vol. 1470, pp. 157–164. Springer, Heidelberg (1998). https://​doi.​org/​10.​1007/​BFb0057848CrossRef
10.
Zurück zum Zitat Hill, J.M., et al.: BSPlib: the BSP programming library. Parallel Comput. 24(14), 1947–1980 (1998)CrossRef Hill, J.M., et al.: BSPlib: the BSP programming library. Parallel Comput. 24(14), 1947–1980 (1998)CrossRef
11.
Zurück zum Zitat Hinsen, K.: High-level parallel software development with python and bsp. Parallel Process. Lett. 13(03), 473–484 (2003)MathSciNetCrossRef Hinsen, K.: High-level parallel software development with python and bsp. Parallel Process. Lett. 13(03), 473–484 (2003)MathSciNetCrossRef
12.
Zurück zum Zitat Hinsen, K.: Parallel scripting with python. Comput. Sci. Eng. 9(6), 82–89 (2007)CrossRef Hinsen, K.: Parallel scripting with python. Comput. Sci. Eng. 9(6), 82–89 (2007)CrossRef
13.
Zurück zum Zitat Hinsen, K., Sadron, R.C.: Parallel programming with BSP in python. Technical report, Centre de Biophysique Moléculaire (2000) Hinsen, K., Sadron, R.C.: Parallel programming with BSP in python. Technical report, Centre de Biophysique Moléculaire (2000)
15.
Zurück zum Zitat Malewicz, G., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 135–146. ACM (2010) Malewicz, G., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 135–146. ACM (2010)
16.
Zurück zum Zitat Seo, S., Yoon, E.J., Kim, J., Jin, S., Kim, J.-S., Maeng, S.: HAMA: an efficient matrix computation with the mapreduce framework. In: 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pp. 721–726. IEEE (2010) Seo, S., Yoon, E.J., Kim, J., Jin, S., Kim, J.-S., Maeng, S.: HAMA: an efficient matrix computation with the mapreduce framework. In: 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pp. 721–726. IEEE (2010)
17.
Zurück zum Zitat Suijlen, W.J.: BsponMPI: an implementation of the BSPlib standard on top of MPI. Version 0.3 (2010) Suijlen, W.J.: BsponMPI: an implementation of the BSPlib standard on top of MPI. Version 0.3 (2010)
18.
Zurück zum Zitat Valiant, L.G.: A bridging model for parallel computation. Commun. ACM 33(8), 103–111 (1990)CrossRef Valiant, L.G.: A bridging model for parallel computation. Commun. ACM 33(8), 103–111 (1990)CrossRef
19.
Zurück zum Zitat Valiant, L.G.: Why BSP computers? [bulk-synchronous parallel computers]. In: Proceedings of Seventh International Parallel Processing Symposium, pp. 2–5. IEEE (1993) Valiant, L.G.: Why BSP computers? [bulk-synchronous parallel computers]. In: Proceedings of Seventh International Parallel Processing Symposium, pp. 2–5. IEEE (1993)
20.
Zurück zum Zitat Zhang, Z., Li, G., Toh, K.-C., Sung, W.-K.: 3D chromosome modeling with semi-definite programming and HI-C data. J. Comput. Biol. 20(11), 831–846 (2013)MathSciNetCrossRef Zhang, Z., Li, G., Toh, K.-C., Sung, W.-K.: 3D chromosome modeling with semi-definite programming and HI-C data. J. Comput. Biol. 20(11), 831–846 (2013)MathSciNetCrossRef
Metadaten
Titel
Data Exchange Engine for Parallel Computing and Its Application to 3D Chromosome Modelling
verfasst von
Xiaoling Zhang
Yao Lu
Junfeng Wu
Yongrui Zhang
Copyright-Jahr
2020
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-15-2810-1_41