Skip to main content

2019 | OriginalPaper | Buchkapitel

Comparing the Efficiency of In Situ Visualization Paradigms at Scale

verfasst von : James Kress, Matthew Larsen, Jong Choi, Mark Kim, Matthew Wolf, Norbert Podhorszki, Scott Klasky, Hank Childs, David Pugmire

Erschienen in: High Performance Computing

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This work compares the two major paradigms for doing in situ visualization: in-line, where the simulation and visualization share the same resources, and in-transit, where simulation and visualization are given dedicated resources. Our runs vary many parameters, including simulation cycle time, visualization frequency, and dedicated resources, to study how tradeoffs change over configuration. In particular, we consider simulations as large as 1,024 nodes (16,384 cores) and dedicated visualization resources with as many as 512 nodes (8,192 cores). We draw conclusions about when each paradigm is superior, such as in-line being superior when the simulation cycle time is very fast. Surprisingly, we also find that in-transit can minimize the total resources consumed for some configurations, since it can cause the visualization routines to require fewer overall resources when they run at lower concurrency. For example, one of our scenarios finds that allocating 25% more resources for visualization allows the simulation to run 61% faster than its in-line comparator. Finally, we explore various models for quantifying the cost for each paradigm, and consider transition points when one paradigm is superior to the other. Our contributions inform design decisions for simulation scientists when performing in situ visualization.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Adhinarayanan, V., Feng, W.C., Rogers, D., Ahrens, J., Pakin, S.: Characterizing and modeling power and energy for extreme-scale in-situ visualization. In: IEEE Parallel and Distributed Processing Symposium (IPDPS), pp. 978–987 (2017) Adhinarayanan, V., Feng, W.C., Rogers, D., Ahrens, J., Pakin, S.: Characterizing and modeling power and energy for extreme-scale in-situ visualization. In: IEEE Parallel and Distributed Processing Symposium (IPDPS), pp. 978–987 (2017)
3.
Zurück zum Zitat Ahern, S., et al.: Scientific Discovery at the Exascale: Report for the DOE ASCR Workshop on Exascale Data Management, Analysis, and Visualization, July 2011 Ahern, S., et al.: Scientific Discovery at the Exascale: Report for the DOE ASCR Workshop on Exascale Data Management, Analysis, and Visualization, July 2011
4.
Zurück zum Zitat Ahrens, J., Geveci, B., Law, C.: ParaView: an end-user tool for large-data visualization. In: The Visualization Handbook, pp. 717–731 (2005)CrossRef Ahrens, J., Geveci, B., Law, C.: ParaView: an end-user tool for large-data visualization. In: The Visualization Handbook, pp. 717–731 (2005)CrossRef
5.
Zurück zum Zitat Ayachit, U., et al.: The SENSEI generic in situ interface. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 40–44, November 2016 Ayachit, U., et al.: The SENSEI generic in situ interface. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 40–44, November 2016
6.
Zurück zum Zitat Ayachit, U., et al.: ParaView catalyst: enabling in situ data analysis and visualization. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 25–29 (2015) Ayachit, U., et al.: ParaView catalyst: enabling in situ data analysis and visualization. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 25–29 (2015)
7.
Zurück zum Zitat Ayachit, U., et al.: Performance analysis, design considerations, and applications of extreme-scale In Situ infrastructures. In: ACM/IEEE Conference for High Performance Computing, Networking, Storage and Analysis (SC16), November 2016 Ayachit, U., et al.: Performance analysis, design considerations, and applications of extreme-scale In Situ infrastructures. In: ACM/IEEE Conference for High Performance Computing, Networking, Storage and Analysis (SC16), November 2016
8.
Zurück zum Zitat Bauer, A.C., et al.: In Situ methods, infrastructures, and applications on high performance computing platforms, a state-of-the-art (STAR) report. Comput. Graph. Forum 35(3), 577 (2016). Proceedings of Eurovis 2016 Bauer, A.C., et al.: In Situ methods, infrastructures, and applications on high performance computing platforms, a state-of-the-art (STAR) report. Comput. Graph. Forum 35(3), 577 (2016). Proceedings of Eurovis 2016
9.
Zurück zum Zitat Bennett, J.C., et al.: Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 49:1–49:9 (2012) Bennett, J.C., et al.: Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 49:1–49:9 (2012)
10.
Zurück zum Zitat Childs, H., et al.: A contract-based system for large data visualization. In: Proceedings of IEEE Visualization 2005, pp. 190–198 (2005) Childs, H., et al.: A contract-based system for large data visualization. In: Proceedings of IEEE Visualization 2005, pp. 190–198 (2005)
11.
Zurück zum Zitat Childs, H., et al.: Extreme scaling of production visualization software on diverse architectures. IEEE Comput. Graph. Appl. (CG&A) 30(3), 22–31 (2010)CrossRef Childs, H., et al.: Extreme scaling of production visualization software on diverse architectures. IEEE Comput. Graph. Appl. (CG&A) 30(3), 22–31 (2010)CrossRef
12.
Zurück zum Zitat Dayal, J., et al.: Flexpath: type-based publish/subscribe system for large-scale science analytics. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) (2014) Dayal, J., et al.: Flexpath: type-based publish/subscribe system for large-scale science analytics. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) (2014)
13.
Zurück zum Zitat Docan, C., et al.: Dataspaces: an interaction and coordination framework for coupled simulation workflows. Cluster Comput. 15(2), 163–181 (2012)CrossRef Docan, C., et al.: Dataspaces: an interaction and coordination framework for coupled simulation workflows. Cluster Comput. 15(2), 163–181 (2012)CrossRef
14.
Zurück zum Zitat Dorier, M., et al.: Damaris/viz: a nonintrusive, adaptable and user-friendly in situ visualization framework. In: IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV), pp. 67–75, October 2013 Dorier, M., et al.: Damaris/viz: a nonintrusive, adaptable and user-friendly in situ visualization framework. In: IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV), pp. 67–75, October 2013
15.
Zurück zum Zitat Gamell, M., et al.: Exploring power behaviors and trade-offs of in-situ data analytics. In: International Conference on High Performance Computing, Networking, Storage and Analysis (SC), pp. 1–12 (2013) Gamell, M., et al.: Exploring power behaviors and trade-offs of in-situ data analytics. In: International Conference on High Performance Computing, Networking, Storage and Analysis (SC), pp. 1–12 (2013)
17.
Zurück zum Zitat Kress, J., et al.: Loosely coupled in situ visualization: a perspective on why it’s here to stay. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 1–6, November 2015 Kress, J., et al.: Loosely coupled in situ visualization: a perspective on why it’s here to stay. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 1–6, November 2015
18.
Zurück zum Zitat Larsen, M., et al.: The ALPINE in situ infrastructure: ascending from the ashes of strawman. In: Workshop on In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 42–46 (2017) Larsen, M., et al.: The ALPINE in situ infrastructure: ascending from the ashes of strawman. In: Workshop on In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 42–46 (2017)
19.
Zurück zum Zitat Liu, Q., et al.: Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks. Concurrency Comput. Pract. Experience 26(7), 1453–1473 (2014)CrossRef Liu, Q., et al.: Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks. Concurrency Comput. Pract. Experience 26(7), 1453–1473 (2014)CrossRef
20.
Zurück zum Zitat Mallinson, A., et al.: Cloverleaf: preparing hydrodynamics codes for exascale. The Cray User Group (2013) Mallinson, A., et al.: Cloverleaf: preparing hydrodynamics codes for exascale. The Cray User Group (2013)
21.
Zurück zum Zitat Moreland, K., et al.: VTK-m: accelerating the visualization toolkit for massively threaded architectures. Comput. Graph. Appl. 36(3), 48–58 (2016)CrossRef Moreland, K., et al.: VTK-m: accelerating the visualization toolkit for massively threaded architectures. Comput. Graph. Appl. 36(3), 48–58 (2016)CrossRef
22.
Zurück zum Zitat Oldfield, R.A., Moreland, K., Fabian, N., Rogers, D.: Evaluation of methods to integrate analysis into a large-scale shock shock physics code. In: Proceedings of the 28th ACM International Conference on Supercomputing, pp. 83–92. ACM (2014) Oldfield, R.A., Moreland, K., Fabian, N., Rogers, D.: Evaluation of methods to integrate analysis into a large-scale shock shock physics code. In: Proceedings of the 28th ACM International Conference on Supercomputing, pp. 83–92. ACM (2014)
23.
Zurück zum Zitat Parker, S., Johnson, C.: SCIRun: a scientific programming environment for computational steering. In: ACM/IEEE Conference on Supercomputing, p. 52 (1995) Parker, S., Johnson, C.: SCIRun: a scientific programming environment for computational steering. In: ACM/IEEE Conference on Supercomputing, p. 52 (1995)
24.
Zurück zum Zitat Peterka, T., Ma, K.L.: Parallel image compositing methods. In: High Performance Visualization: Enabling Extreme-Scale Scientific Insight (2012) Peterka, T., Ma, K.L.: Parallel image compositing methods. In: High Performance Visualization: Enabling Extreme-Scale Scientific Insight (2012)
25.
Zurück zum Zitat Rodero, I., et al.: Evaluation of in-situ analysis strategies at scale for power efficiency and scalability. In: IEEE/ACM Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 156–164 (2016) Rodero, I., et al.: Evaluation of in-situ analysis strategies at scale for power efficiency and scalability. In: IEEE/ACM Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 156–164 (2016)
26.
Zurück zum Zitat Whitlock, B., Favre, J., Meredith, J.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Proceedings of the 11th Eurographics Conference on Parallel Graphics and Visualization, pp. 101–109 (2011) Whitlock, B., Favre, J., Meredith, J.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Proceedings of the 11th Eurographics Conference on Parallel Graphics and Visualization, pp. 101–109 (2011)
27.
Zurück zum Zitat Zhang, F., et al.: In-memory staging and data-centric task placement for coupled scientific simulation workflows. Concurrency Comput. Pract. Experience 29(12), e4147 (2017)CrossRef Zhang, F., et al.: In-memory staging and data-centric task placement for coupled scientific simulation workflows. Concurrency Comput. Pract. Experience 29(12), e4147 (2017)CrossRef
Metadaten
Titel
Comparing the Efficiency of In Situ Visualization Paradigms at Scale
verfasst von
James Kress
Matthew Larsen
Jong Choi
Mark Kim
Matthew Wolf
Norbert Podhorszki
Scott Klasky
Hank Childs
David Pugmire
Copyright-Jahr
2019
DOI
https://doi.org/10.1007/978-3-030-20656-7_6

Premium Partner