nach oben

Erschienen in:

2019 | OriginalPaper | Buchkapitel

Supporting Efficient Execution of Workflows on Everest Platform

verfasst von : Oleg Sukhoroslov

Erschienen in: Supercomputing

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Workflows is an important class of parallel applications consisting of multiple tasks with control or data dependencies between them. Such applications are widely used for automation of computational and data processing pipelines in science and technology. The paper considers the execution of workflows on Everest, a web-based distributed computing platform. Two different approaches for workflow description and execution are presented and compared. The proposed solutions and optimizations address common problems related to the efficient execution of workflows on distributed computing resources. The results of experimental study demonstrate the effectiveness of proposed optimizations.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel SAT-Based Cryptanalysis: From Parallel Computing to Volunteer Computing

Nächstes Kapitel Start-up and the Results of the Volunteer Computing Project RakeSearch

Everest. http://everest.distcomp.org/

Everest Python API. https://gitlab.com/everest/python-api

Afanasiev, A., Sukhoroslov, O., Voloshinov, V.: MathCloud: publication and reuse of scientific applications as RESTful web services. In: Malyshkin, V. (ed.) PaCT 2013. LNCS, vol. 7979, pp. 394–408. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39958-9_36CrossRef

Albrecht, M., Donnelly, P., Bui, P., Thain, D.: Makeflow: a portable abstraction for data intensive computing on clusters, clouds, and grids. In: Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies, p. 1. ACM (2012)

Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of scientific workflows. In: 2008 Third Workshop on Workflows in Support of Large-Scale Science, pp. 1–10, November 2008

Deelman, E., Gannon, D., Shields, M., Taylor, I.: Workflows and e-science: an overview of workflow system features and capabilities. Futur. Gener. Comput. Syst. 25(5), 528–540 (2009)CrossRef

Deelman, E., et al.: Pegasus, a workflow management system for science automation. Futur. Gener. Comput. Syst. 46, 17–35 (2015)CrossRef

Fahringer, T., et al.: ASKALON: a grid application development and computing environment. In: The 6th IEEE/ACM International Workshop on Grid Computing, pp. 10–pp. IEEE (2005)

Glatard, T., Montagnat, J., Lingrand, D., Pennec, X.: Flexible and efficient workflow deployment of data-intensive applications on grids with moteur. Int. J. High Perform. Comput. Appl. 22(3), 347–360 (2008)CrossRef

10.

Goecks, J., Nekrutenko, A., Taylor, J.: Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biol. 11(8), R86 (2010)CrossRef

11.

Kacsuk, P., et al.: WS-PGRADE/gUSE generic DCI gateway framework for a large variety of user communities. J. Grid Comput. 10(4), 601–630 (2012)CrossRef

12.

Smirnov, S., Sukhoroslov, O., Volkov, S.: Integration and combined use of distributed computing resources with Everest. J. Grid Comput. 101, 359–368 (2016)

13.

Sukhoroslov, O., Volkov, S., Afanasiev, A.: A web-based platform for publication and distributed execution of computing applications. In: 2015 14th International Symposium on Parallel and Distributed Computing (ISPDC), pp. 175–184, June 2015

14.

Sukhoroslov, O.: Integration of Everest platform with BOINC-based desktop grids. In: Ivashko, E., Rumyantsev, A. (eds.) Proceedings of the Third International Conference BOINC: FAST 2017, Petrozavodsk, Russia, pp. 102–107 (2017)

15.

Sukhoroslov, O., Nazarenko, A., Aleksandrov, R.: An experimental study of scheduling algorithms for many-task applications. J. Supercomput. 75, 7857–7871 (2018). https://doi.org/10.1007/s11227-018-2553-9CrossRef

16.

Sukhoroslov, O.: Supporting efficient execution of many-task applications with Everest. In: Proceedings of the VIII International Conference “Distributed Computing and Grid-Technologies in Science and Education” (GRID 2018), pp. 266–270 (2018)

17.

Taylor, I., Shields, M., Wang, I., Harrison, A.: The Triana workflow environment: architecture and applications. In: Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M. (eds.) Workflows for e-Science, pp. 320–339. Springer, Heidelberg (2007). https://doi.org/10.1007/978-1-84628-757-2_20CrossRef

18.

Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M.: Workflows for e-Science: sCientific Workflows for Grids. Springer, Heidelberg (2014). https://doi.org/10.1007/978-1-84628-757-2. IncorporatedCrossRef

19.

Topcuoglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)CrossRef

20.

Volkov, S., Sukhoroslov, O.: A generic web service for running parameter sweep experiments in distributed computing environment. Procedia Comput. Sci. 66, 477–486 (2015)CrossRef

21.

Volkov, S., Sukhoroslov, O.: Simplifying the use of clouds for scientific computing with Everest. Procedia Comput. Sci. 119, 112–120 (2017)CrossRef

22.

Voloshinov, V., Smirnov, S., Sukhoroslov, O.: Implementation and use of coarse-grained parallel branch-and-bound in Everest distributed environment. Procedia Comput. Sci. 108, 1532–1541 (2017)CrossRef

23.

Wieczorek, M., Prodan, R., Fahringer, T.: Scheduling of scientific workflows in the askalon grid environment. ACM SIGMOD Rec. 34(3), 56–62 (2005)CrossRef

24.

Wolstencroft, K., et al.: The Taverna workflow suite: designing and executing workflows of web services on the desktop, web or in the cloud. Nucl. Acids Res. 41(W1), W557–W561 (2013)CrossRef

25.

Wu, F., Wu, Q., Tan, Y.: Workflow scheduling in cloud: a survey. J. Supercomput. 71(9), 3373–3418 (2015)CrossRef

26.

Yu, J., Buyya, R., Ramamohanarao, K.: Workflow scheduling algorithms for grid computing. In: Xhafa, F., Abraham, A. (eds.) Metaheuristics for Scheduling in Distributed Computing Environments, pp. 173–214. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69277-5_7CrossRefMATH

Titel: Supporting Efficient Execution of Workflows on Everest Platform
verfasst von: Oleg Sukhoroslov
Verlag: Springer International Publishing
Buch: Supercomputing
Print ISBN: 978-3-030-36591-2

Electronic ISBN: 978-3-030-36592-9

Copyright-Jahr: 2019
DOI: https://doi.org/10.1007/978-3-030-36592-9_58

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner