Skip to main content

2019 | OriginalPaper | Buchkapitel

Analyzing Communication Features and Community Structure of HPC Applications

verfasst von : Manfred Calvo, Diego Jiménez, Esteban Meneses

Erschienen in: High Performance Computing

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

A few exascale machines are scheduled to become operational in the next couple of years. Reaching such achievement required the HPC community to overcome obstacles in programmability, power management, memory hierarchy, and reliability. Similar challenges are to be faced in the pursuit of greater performance gains. In particular, design of interconnects stands out as a major hurdle. Computer networks for extreme-scale system will need a deeper understanding of the communication characteristics of applications that will run on those systems. We analyzed a set of nine representative HPC applications and created a catalog of well-defined communication patterns that constitute building blocks for modern scientific codes. Furthermore, we found little difference between popular community-detection algorithms, which tend to form few but relatively big communities.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
3.
Zurück zum Zitat Barrett, R., et al.: On the role of co-design in high performance computing, vol. 24, pp. 141–155 (2013) Barrett, R., et al.: On the role of co-design in high performance computing, vol. 24, pp. 141–155 (2013)
4.
Zurück zum Zitat Blondel, V.D., Guillaume, J.L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008(10), P10008 (2008)CrossRef Blondel, V.D., Guillaume, J.L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008(10), P10008 (2008)CrossRef
5.
Zurück zum Zitat Brightwell, R., Barrett, B.W., Hemmert, K.S., Underwood, K.D.: Challenges for high-performance networking for exascale computing. In: 2010 Proceedings of 19th International Conference on Computer Communications and Networks, pp. 1–6, August 2010 Brightwell, R., Barrett, B.W., Hemmert, K.S., Underwood, K.D.: Challenges for high-performance networking for exascale computing. In: 2010 Proceedings of 19th International Conference on Computer Communications and Networks, pp. 1–6, August 2010
6.
Zurück zum Zitat Clauset, A., Newman, M.E.J., Moore, C.: Finding community structure in very large networks. Phys. Rev. E 70, 066111 (2004)CrossRef Clauset, A., Newman, M.E.J., Moore, C.: Finding community structure in very large networks. Phys. Rev. E 70, 066111 (2004)CrossRef
8.
Zurück zum Zitat Dongarra, J., et al.: The international exascale software project roadmap (2011) Dongarra, J., et al.: The international exascale software project roadmap (2011)
9.
Zurück zum Zitat Girvan, M., Newman, M.E.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. 99(12), 7821–7826 (2002)MathSciNetCrossRef Girvan, M., Newman, M.E.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. 99(12), 7821–7826 (2002)MathSciNetCrossRef
10.
Zurück zum Zitat Heroux, M.A., et al.: Improving performance via mini-applications. Technical report SAND2009-5574, Sandia National Laboratories (2009) Heroux, M.A., et al.: Improving performance via mini-applications. Technical report SAND2009-5574, Sandia National Laboratories (2009)
11.
Zurück zum Zitat Hoefler, T., Snir, M.: Generic topology mapping strategies for large-scale parallel architectures. In: Proceedings of the 2011 ACM International Conference on Supercomputing (ICS 2011), pp. 75–85. ACM, June 2011 Hoefler, T., Snir, M.: Generic topology mapping strategies for large-scale parallel architectures. In: Proceedings of the 2011 ACM International Conference on Supercomputing (ICS 2011), pp. 75–85. ACM, June 2011
12.
Zurück zum Zitat Hoefler, T., Jeannot, E., Mercier, G.: An overview of process mapping techniques and algorithms in high-performance computing (2014) Hoefler, T., Jeannot, E., Mercier, G.: An overview of process mapping techniques and algorithms in high-performance computing (2014)
13.
Zurück zum Zitat Kogge, P., et al.: Exascale computing study: technology challenges in achieving exascale systems (2008) Kogge, P., et al.: Exascale computing study: technology challenges in achieving exascale systems (2008)
14.
Zurück zum Zitat Leskovec, J., Lang, K.J., Mahoney, M.: Empirical comparison of algorithms for network community detection. In: Proceedings of the 19th International Conference on World Wide Web, pp. 631–640 (2010) Leskovec, J., Lang, K.J., Mahoney, M.: Empirical comparison of algorithms for network community detection. In: Proceedings of the 19th International Conference on World Wide Web, pp. 631–640 (2010)
16.
Zurück zum Zitat Newman, M.E.J.: Fast algorithm for detecting community structure in networks. Phys. Rev. E 69, 066133 (2004)CrossRef Newman, M.E.J.: Fast algorithm for detecting community structure in networks. Phys. Rev. E 69, 066133 (2004)CrossRef
17.
Zurück zum Zitat Newman, M.E.J.: Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74, 036104 (2006)MathSciNetCrossRef Newman, M.E.J.: Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74, 036104 (2006)MathSciNetCrossRef
18.
Zurück zum Zitat Newman, M.E.J., Girvan, M.: Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113 (2004)CrossRef Newman, M.E.J., Girvan, M.: Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113 (2004)CrossRef
19.
Zurück zum Zitat Radicchi, F., Castellano, C., Cecconi, F., Loreto, V., Parisi, D.: Defining and identifying communities in networks. Proc. Natl. Acad. Sci. U.S.A. 101(9), 2658–2663 (2004)CrossRef Radicchi, F., Castellano, C., Cecconi, F., Loreto, V., Parisi, D.: Defining and identifying communities in networks. Proc. Natl. Acad. Sci. U.S.A. 101(9), 2658–2663 (2004)CrossRef
20.
Zurück zum Zitat Raponi, P.G., Petrini, F., Walkup, R., Checconi, F.: Characterization of the communication patterns of scientific applications on blue gene/p. In: 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum (IPDPSW), pp. 1017–1024 (2011) Raponi, P.G., Petrini, F., Walkup, R., Checconi, F.: Characterization of the communication patterns of scientific applications on blue gene/p. In: 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum (IPDPSW), pp. 1017–1024 (2011)
21.
Zurück zum Zitat Riesen, R.: Communication patterns [message-passing patterns]. In: 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, 8 pp. IEEE (2006) Riesen, R.: Communication patterns [message-passing patterns]. In: 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, 8 pp. IEEE (2006)
22.
Zurück zum Zitat Ropars, T., Guermouche, A., Uçar, B., Meneses, E., Kalé, L.V., Cappello, F.: On the use of cluster-based partial message logging to improve fault tolerance for MPI HPC applications. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011. LNCS, vol. 6852, pp. 567–578. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23400-2_53CrossRef Ropars, T., Guermouche, A., Uçar, B., Meneses, E., Kalé, L.V., Cappello, F.: On the use of cluster-based partial message logging to improve fault tolerance for MPI HPC applications. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011. LNCS, vol. 6852, pp. 567–578. Springer, Heidelberg (2011). https://​doi.​org/​10.​1007/​978-3-642-23400-2_​53CrossRef
23.
Zurück zum Zitat Roth, P.C., Meredith, J.S., Vetter, J.S.: Automated characterization of parallel application communication patterns. In: Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, pp. 73–84. ACM (2015) Roth, P.C., Meredith, J.S., Vetter, J.S.: Automated characterization of parallel application communication patterns. In: Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, pp. 73–84. ACM (2015)
25.
Zurück zum Zitat Vetter, J.S., Mueller, F.: Communication characteristics of large-scale scientific applications for contemporary cluster architectures. J. Parallel Distrib. Comput. 63(9), 853–865 (2003)CrossRef Vetter, J.S., Mueller, F.: Communication characteristics of large-scale scientific applications for contemporary cluster architectures. J. Parallel Distrib. Comput. 63(9), 853–865 (2003)CrossRef
26.
Zurück zum Zitat Vetter, J.S., Yoo, A.: An empirical performance evaluation of scalable scientific applications. In: ACM/IEEE 2002 Conference on Supercomputing, p. 16. IEEE (2002) Vetter, J.S., Yoo, A.: An empirical performance evaluation of scalable scientific applications. In: ACM/IEEE 2002 Conference on Supercomputing, p. 16. IEEE (2002)
28.
Zurück zum Zitat Xue, R., et al.: MPIWiz: subgroup reproducible replay of MPI applications. In: Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2009, pp. 251–260. ACM, New York (2009) Xue, R., et al.: MPIWiz: subgroup reproducible replay of MPI applications. In: Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2009, pp. 251–260. ACM, New York (2009)
29.
Zurück zum Zitat Yang, Z., Algesheimer, R., Tessone, C.J.: A comparative analysis of community detection algorithms on artificial networks. Sci. Rep. 6, 30750 (2016)CrossRef Yang, Z., Algesheimer, R., Tessone, C.J.: A comparative analysis of community detection algorithms on artificial networks. Sci. Rep. 6, 30750 (2016)CrossRef
Metadaten
Titel
Analyzing Communication Features and Community Structure of HPC Applications
verfasst von
Manfred Calvo
Diego Jiménez
Esteban Meneses
Copyright-Jahr
2019
DOI
https://doi.org/10.1007/978-3-030-16205-4_18