Skip to main content

2018 | OriginalPaper | Buchkapitel

ClustMap: A Topology-Aware MPI Process Placement Algorithm for Multi-core Clusters

verfasst von : K. B. Manwade, D. B. Kulkarni

Erschienen in: Intelligent Computing and Information and Communication

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Many high-performance computing applications are using MPI (Message Passing Interface) for communication. The performance of MPI library affects the performance of MPI applications. Various techniques like communication latency reduction, increasing bandwidth, and increasing scalability are available for improving the performance of message passing. In multi-core cluster environment, the communication latency can be further reduced by topology-aware process placement. This technique involves three steps: finding communication pattern of MPI application (application topology), finding architecture details of underlying multi-core cluster (system topology), and mapping processes to cores. In this paper, we have proposed novel “ClustMap” algorithm for the third step. By using this algorithm, both system and application topologies are mapped. The experimental results show that the proposed algorithm outperforms over existing process placement techniques.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Balaji Venu, “Multi core processors: An overview”, Department of Electrical Engineering and Electronics, University of Liverpool, Liverpool, UK, (2010). Balaji Venu, “Multi core processors: An overview”, Department of Electrical Engineering and Electronics, University of Liverpool, Liverpool, UK, (2010).
2.
Zurück zum Zitat Dennis Abts and John Kim, “High Performance Datacenter Networks: Architectures, Algorithms, and Opportunities”, Synthesis Lectures on Computer Architecture, Book series, Pages 1–115, Vol. 6, No. 1, Morgan Claypool Publishers, (2011). Dennis Abts and John Kim, “High Performance Datacenter Networks: Architectures, Algorithms, and Opportunities”, Synthesis Lectures on Computer Architecture, Book series, Pages 1–115, Vol. 6, No. 1, Morgan Claypool Publishers, (2011).
3.
Zurück zum Zitat Mohammad Javad Rashti, Jonathan Green, Pavan Balaji, Ahmad Afsahi and William Gropp, “Multi-core and Network Aware MPI Topology Functions, EuroMPI’11, ISBN: 978-3-642-24448-3 (2011), pp. 50–60. Mohammad Javad Rashti, Jonathan Green, Pavan Balaji, Ahmad Afsahi and William Gropp, “Multi-core and Network Aware MPI Topology Functions, EuroMPI’11, ISBN: 978-3-642-24448-3 (2011), pp. 50–60.
4.
Zurück zum Zitat Joshua Hursey, Jerey M. Squyres and Terry Dontje, “Locality-Aware Parallel Process Mapping for Multi-Core HPC Systems”, IEEE International Conference on Cluster Computing, (2011). Joshua Hursey, Jerey M. Squyres and Terry Dontje, “Locality-Aware Parallel Process Mapping for Multi-Core HPC Systems”, IEEE International Conference on Cluster Computing, (2011).
5.
Zurück zum Zitat B. Brandfass, T. Alrutz and T. Gerhold, “Rank reordering for MPI communication optimization”, 23rd Int. Conf. on Parallel Fluid Dynamics, (2013), pp. 372–380. B. Brandfass, T. Alrutz and T. Gerhold, “Rank reordering for MPI communication optimization”, 23rd Int. Conf. on Parallel Fluid Dynamics, (2013), pp. 372–380.
6.
Zurück zum Zitat Torsten Hoefler, Emmanuel Jeannot, Guillaume Mercier, “An Overview of Process Mapping Techniques and Algorithms in High-Performance Computing”, High Performance Computing on Complex Environments, Wiley, pp. 75–94, 2014, 978-1-118-71205-4 <hal-00921626>, (2014) Torsten Hoefler, Emmanuel Jeannot, Guillaume Mercier, “An Overview of Process Mapping Techniques and Algorithms in High-Performance Computing”, High Performance Computing on Complex Environments, Wiley, pp. 75–94, 2014, 978-1-118-71205-4 <hal-00921626>, (2014)
7.
Zurück zum Zitat Alberto Fernndez and Sergio Gmez, “Solving Non-Uniqueness in Agglomerative Hierarchical Clustering Using Multi-dendrograms”, Journal of Classification 25:43-65, ISSN: 0176–4268, (2008). Alberto Fernndez and Sergio Gmez, “Solving Non-Uniqueness in Agglomerative Hierarchical Clustering Using Multi-dendrograms”, Journal of Classification 25:43-65, ISSN: 0176–4268, (2008).
9.
Zurück zum Zitat B. W. Kernighan and S. Lin, “An efficient heuristic procedure for partitioning graphs”, Bell System Technical Journal, 59 (1970), pp. 291–307. B. W. Kernighan and S. Lin, “An efficient heuristic procedure for partitioning graphs”, Bell System Technical Journal, 59 (1970), pp. 291–307.
10.
Zurück zum Zitat T. Hoeer and M. Snir, “Generic topology mapping strategies for large-scale parallel architectures”, Int. Conf. on Supercomputing, (2011), pp. 75–84. T. Hoeer and M. Snir, “Generic topology mapping strategies for large-scale parallel architectures”, Int. Conf. on Supercomputing, (2011), pp. 75–84.
11.
Zurück zum Zitat T. Hatazaki, “Rank reordering strategy for MPI topology creation functions”, Recent Advances in Parallel Virtual Machine and Message Passing Interface, EuroPVM/MPI 1998, Lecture Notes in Computer Science, vol 1497, pp. 188–195, (1998). T. Hatazaki, “Rank reordering strategy for MPI topology creation functions”, Recent Advances in Parallel Virtual Machine and Message Passing Interface, EuroPVM/MPI 1998, Lecture Notes in Computer Science, vol 1497, pp. 188–195, (1998).
12.
Zurück zum Zitat S. H. Bokhari, “On the mapping problem”, IEEE Transactions on Computers, Volume 30 Issue 3, Pages 207–214, ISSN: 0018-9340, (1981). S. H. Bokhari, “On the mapping problem”, IEEE Transactions on Computers, Volume 30 Issue 3, Pages 207–214, ISSN: 0018-9340, (1981).
13.
Zurück zum Zitat S. Y. Lee and J. K. Aggarwal, “A mapping strategy for parallel processing”, IEEE Transactions on Computers, Volume 36 Issue 4, Pages 433-442, ISSN: 0018–9340, (1987). S. Y. Lee and J. K. Aggarwal, “A mapping strategy for parallel processing”, IEEE Transactions on Computers, Volume 36 Issue 4, Pages 433-442, ISSN: 0018–9340, (1987).
14.
Zurück zum Zitat C. Sudheer and A. Srinivasan, “Optimization of the hop-byte metric for effective topology-aware mapping”, 19th Int. Conf. on HPC, (2012), pp. 19. C. Sudheer and A. Srinivasan, “Optimization of the hop-byte metric for effective topology-aware mapping”, 19th Int. Conf. on HPC, (2012), pp. 19.
15.
Zurück zum Zitat P. Balaji, R. Gupta, A. Vishnu, and P. H. Beckman, “Mapping communication layouts to network hardware characteristics on massive-scale Blue Gene systems”, Journal of Computer Science - Research and Development, Volume 26 Issue 3-4, Pages 247–256, ISSN:1865-2034 (2011). P. Balaji, R. Gupta, A. Vishnu, and P. H. Beckman, “Mapping communication layouts to network hardware characteristics on massive-scale Blue Gene systems”, Journal of Computer Science - Research and Development, Volume 26 Issue 3-4, Pages 247–256, ISSN:1865-2034 (2011).
16.
Zurück zum Zitat B. E. Smith and B. Bode, “Performance effects of node mappings on the IBM Blue-Gene/L machine”, Euro-Par 2005. Lecture Notes in Computer Science, vol 3648, Pages 1005–1013, ISBN: 978-3-540-28700-1, (2005). B. E. Smith and B. Bode, “Performance effects of node mappings on the IBM Blue-Gene/L machine”, Euro-Par 2005. Lecture Notes in Computer Science, vol 3648, Pages 1005–1013, ISBN: 978-3-540-28700-1, (2005).
17.
Zurück zum Zitat J. L. Traff, “Implementing the MPI process topology mechanism”, Proceedings of the 2002 ACM/IEEE conf. on Supercomputing, IEEE Computer Society Press, (2002), pp. 114. J. L. Traff, “Implementing the MPI process topology mechanism”, Proceedings of the 2002 ACM/IEEE conf. on Supercomputing, IEEE Computer Society Press, (2002), pp. 114.
18.
Zurück zum Zitat Guillaume Mercier, Jerome Clet-Ortega, “Towards an efficient process placement policy for MPI applications in multi-core environments”, Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2009, pp. 104–115, ISBN: 978-3-642-03769-6 (2009). Guillaume Mercier, Jerome Clet-Ortega, “Towards an efficient process placement policy for MPI applications in multi-core environments”, Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2009, pp. 104–115, ISBN: 978-3-642-03769-6 (2009).
19.
Zurück zum Zitat Henri Casanova, Arnaud Giersch, Arnaud Legrand, Martin Quinson, Frederic Suter, “Versatile, Scalable, and Accurate Simulation of Distributed Applications and Platforms”, Journal of Parallel and Distributed Computing, Volume 74, Issue 10, pp. 2899–2917, (2014). Henri Casanova, Arnaud Giersch, Arnaud Legrand, Martin Quinson, Frederic Suter, “Versatile, Scalable, and Accurate Simulation of Distributed Applications and Platforms”, Journal of Parallel and Distributed Computing, Volume 74, Issue 10, pp. 2899–2917, (2014).
Metadaten
Titel
ClustMap: A Topology-Aware MPI Process Placement Algorithm for Multi-core Clusters
verfasst von
K. B. Manwade
D. B. Kulkarni
Copyright-Jahr
2018
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-7245-1_8

Premium Partner