Skip to main content

2017 | OriginalPaper | Buchkapitel

Proposing an Architecture for Scientific Workflow Management System in Cloud

verfasst von : Vahab Samandi, Debajyoti Mukhopadhyay

Erschienen in: Computing and Network Sustainability

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

With the growth in IT infrastructure and advances in technologies, workflow scheduling poses many challenging issues for complex applications which require many computing resources. Hence, there is a requirement of a workflow management system adaptable with many cloud environments due to the heterogeneity of resources and applications. In this paper, we have proposed a general workflow management system architecture and a scientific workflow model, followed by a model for monitoring tool in the cloud environment, based on a comprehensive study of literature in cloud computing.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Yu J, Buyya R, Ramamohanarao K (2008) Workflow scheduling algorithms for grid computing. In: Metaheuristics for scheduling in distributed computing environments, volume 146 of the series studies in computational intelligence, pp 173–214 Yu J, Buyya R, Ramamohanarao K (2008) Workflow scheduling algorithms for grid computing. In: Metaheuristics for scheduling in distributed computing environments, volume 146 of the series studies in computational intelligence, pp 173–214
6.
Zurück zum Zitat Vahi K, Harvey I, Samak T, Gunter D, Evans K, Rogers D, Taylor I, Goode M, Silva F, Al-Shakarchi E, Mehta G, Deelman E, Jones A (2013) A case study into using common real-time workflow monitoring infrastructure for scientific workflows. Springer Science + Business Media, Dordrecht Vahi K, Harvey I, Samak T, Gunter D, Evans K, Rogers D, Taylor I, Goode M, Silva F, Al-Shakarchi E, Mehta G, Deelman E, Jones A (2013) A case study into using common real-time workflow monitoring infrastructure for scientific workflows. Springer Science + Business Media, Dordrecht
10.
Zurück zum Zitat Juve G, Deelman E (2011) Scientific workflows in the cloud. in: grids, clouds and virtualization. Comput Commun Netw 71–89 (Springer, London Limited) Juve G, Deelman E (2011) Scientific workflows in the cloud. in: grids, clouds and virtualization. Comput Commun Netw 71–89 (Springer, London Limited)
11.
Zurück zum Zitat Jin H, Ibrahim S, Bell T, Qi L, Cao H, Wu S, Shi X (2010) Tools and technologies for building clouds. In: Cloud computing part of the series computer communications and networks, Springer, London Limited, pp 3–20 Jin H, Ibrahim S, Bell T, Qi L, Cao H, Wu S, Shi X (2010) Tools and technologies for building clouds. In: Cloud computing part of the series computer communications and networks, Springer, London Limited, pp 3–20
12.
Zurück zum Zitat Yu J, Buyya R (2006) Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms. Sci Program 14(3–4):217–230 Yu J, Buyya R (2006) Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms. Sci Program 14(3–4):217–230
13.
Zurück zum Zitat Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 1–46 (Springer Science + Business Media, New York) Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 1–46 (Springer Science + Business Media, New York)
14.
Zurück zum Zitat Cunsolo VD, Distefano S, Puliafito A, Scarpa M (2010) Open and interoperable clouds: the Cloud@Home Way. In: Cloud computing part of the series computer communications and networks. Springer, London Limited, pp 93–111 Cunsolo VD, Distefano S, Puliafito A, Scarpa M (2010) Open and interoperable clouds: the Cloud@Home Way. In: Cloud computing part of the series computer communications and networks. Springer, London Limited, pp 93–111
15.
Zurück zum Zitat Zhang Fan, Cao Junwei, Tan Wei, Khan Samee U, Li Keqin, Zomaya Albert Y (2014) Evolutionary scheduling of dynamic multitasking workloads for big-data analytics in elastic cloud. IEEE Trans Emerg Top Comput 2(3):338–351CrossRef Zhang Fan, Cao Junwei, Tan Wei, Khan Samee U, Li Keqin, Zomaya Albert Y (2014) Evolutionary scheduling of dynamic multitasking workloads for big-data analytics in elastic cloud. IEEE Trans Emerg Top Comput 2(3):338–351CrossRef
16.
Zurück zum Zitat Li X, Song J, Huang B (2015) A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics. Int J Adv Manuf Technol 1–13 (Springer, London) Li X, Song J, Huang B (2015) A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics. Int J Adv Manuf Technol 1–13 (Springer, London)
17.
Zurück zum Zitat Zhang F, Cao J, Hwang K, Wu C (2011) Ordinal optimized scheduling of scientific workflows in elastic compute clouds. In: Third IEEE international conference on cloud computing technology and science, pp 9–17 Zhang F, Cao J, Hwang K, Wu C (2011) Ordinal optimized scheduling of scientific workflows in elastic compute clouds. In: Third IEEE international conference on cloud computing technology and science, pp 9–17
18.
Zurück zum Zitat Varalakshmi P, Ramaswamy A, Balasubramanian A, Vijaykumar P (2011) An optimal workflow based scheduling and resource allocation in cloud. In: Advances in computing and communications volume 190 of the series communications in computer and information science, Springer, Berlin, Heidelberg, pp 411–420 Varalakshmi P, Ramaswamy A, Balasubramanian A, Vijaykumar P (2011) An optimal workflow based scheduling and resource allocation in cloud. In: Advances in computing and communications volume 190 of the series communications in computer and information science, Springer, Berlin, Heidelberg, pp 411–420
19.
Zurück zum Zitat Ho Y-C, Zhao Q-C, Jia Q-S (2007) Ordinal optimization, soft optimization for hard problems. Springer Science + Business Media, New York, USA Ho Y-C, Zhao Q-C, Jia Q-S (2007) Ordinal optimization, soft optimization for hard problems. Springer Science + Business Media, New York, USA
20.
Zurück zum Zitat Cafaro M, Aloisio G (2011) Grids, clouds and virtualization, computer communications and networks. Springer, London Limited, pp 71–91 Cafaro M, Aloisio G (2011) Grids, clouds and virtualization, computer communications and networks. Springer, London Limited, pp 71–91
21.
Zurück zum Zitat Prodan R, Wieczorek M (2010) Bi-criteria scheduling of scientific grid workflows. IEEE Trans Autom Sci Eng 7(2):364–376CrossRef Prodan R, Wieczorek M (2010) Bi-criteria scheduling of scientific grid workflows. IEEE Trans Autom Sci Eng 7(2):364–376CrossRef
22.
Zurück zum Zitat Chen C, Liu J, Wen Y, Chen J (2015) Research on workflow scheduling algorithms in the cloud. In: Process-aware systems, volume 495 of the series communications in computer and information science. Springer, Berlin, Heidelberg, pp 35–48 Chen C, Liu J, Wen Y, Chen J (2015) Research on workflow scheduling algorithms in the cloud. In: Process-aware systems, volume 495 of the series communications in computer and information science. Springer, Berlin, Heidelberg, pp 35–48
23.
Zurück zum Zitat Benoit A, Marchal L, Pineau J-F, Robert Y, Vivien F (2009) Resource-aware allocation strategies for divisible loads on large-scale systems. In: Proceedings of IEEE international parallel and distributed processing symposium (IPDPS), Rome, Italy, pp 1–4 Benoit A, Marchal L, Pineau J-F, Robert Y, Vivien F (2009) Resource-aware allocation strategies for divisible loads on large-scale systems. In: Proceedings of IEEE international parallel and distributed processing symposium (IPDPS), Rome, Italy, pp 1–4
24.
Zurück zum Zitat Zhang F, Cao J, Hwang K, Li K, Khan SU (2015) Adaptive workflow scheduling on cloud computing platforms with iterative ordinal optimization. IEEE Trans Cloud Comput 3(2):156–168 Zhang F, Cao J, Hwang K, Li K, Khan SU (2015) Adaptive workflow scheduling on cloud computing platforms with iterative ordinal optimization. IEEE Trans Cloud Comput 3(2):156–168
25.
Zurück zum Zitat Rodriguez MA, Buyya R (2014) Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans Cloud Comput 2(2):222–234 Rodriguez MA, Buyya R (2014) Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans Cloud Comput 2(2):222–234
26.
Zurück zum Zitat Emeakaroha VC, Maurer M, Stern P, Labaj PP, Brandic I, Kreil DP (2013) Managing and optimizing bioinformatics workflows for data analysis in clouds. J Grid Comput 407–427 (Springer Science + Business Media Dordrecht) Emeakaroha VC, Maurer M, Stern P, Labaj PP, Brandic I, Kreil DP (2013) Managing and optimizing bioinformatics workflows for data analysis in clouds. J Grid Comput 407–427 (Springer Science + Business Media Dordrecht)
27.
Zurück zum Zitat Calheiros RN, Masoumi E, Ranjan R, Buyya R (2014) Workload prediction using ARIMA model and its impact on cloud applications’ QoS. IEEE Trans Cloud Comput 1–11 Calheiros RN, Masoumi E, Ranjan R, Buyya R (2014) Workload prediction using ARIMA model and its impact on cloud applications’ QoS. IEEE Trans Cloud Comput 1–11
28.
Zurück zum Zitat Freund RF et al (1998) Scheduling resources in multi-user, heterogeneous, computing environments with smartnet. In: Proceedings of 7th heterogeneous computing workshop (HCW), Washington, DC, USA, pp 184–199 Freund RF et al (1998) Scheduling resources in multi-user, heterogeneous, computing environments with smartnet. In: Proceedings of 7th heterogeneous computing workshop (HCW), Washington, DC, USA, pp 184–199
29.
Zurück zum Zitat Li H, Buyya R (2007) Model-driven simulation of grid scheduling strategies. In: 3rd IEEE international conference on e-science and grid computing 287–294 Li H, Buyya R (2007) Model-driven simulation of grid scheduling strategies. In: 3rd IEEE international conference on e-science and grid computing 287–294
30.
Zurück zum Zitat Smith J, Siegel HJ, Maciejewski AA (2008) A stochastic model for robust resource allocation in heterogeneous parallel and distributed computing systems. In: Proceedings of IEEE international parallel and distributed processing symposium, Miami, FL, USA, pp 1–5 Smith J, Siegel HJ, Maciejewski AA (2008) A stochastic model for robust resource allocation in heterogeneous parallel and distributed computing systems. In: Proceedings of IEEE international parallel and distributed processing symposium, Miami, FL, USA, pp 1–5
31.
Zurück zum Zitat Lin C, Lu S, Fei X, Chebotko A, Pai D, Lai Z, Fotouhi F, Hua J (2009) A reference architecture for scientific workflow management systems and the VIEW SOA solution. IEEE Trans Serv Comput 2(1):79–92 Lin C, Lu S, Fei X, Chebotko A, Pai D, Lai Z, Fotouhi F, Hua J (2009) A reference architecture for scientific workflow management systems and the VIEW SOA solution. IEEE Trans Serv Comput 2(1):79–92
32.
Zurück zum Zitat Macías M, Guitart J (2012) Client classification policies for SLA negotiation and allocation in shared cloud datacenters. In: Economics of grids, clouds, systems, and services, volume 7150 of the series Lecture Notes in Computer Science pp 90–104 Macías M, Guitart J (2012) Client classification policies for SLA negotiation and allocation in shared cloud datacenters. In: Economics of grids, clouds, systems, and services, volume 7150 of the series Lecture Notes in Computer Science pp 90–104
33.
Zurück zum Zitat Moses J, Iyer R, Illikkal R, Srinivasan S, Aisopos K (2011) Shared resource monitoring and throughput optimization in cloud-computing datacenters. In: IEEE international parallel and distributed processing symposium, pp 1024–1033 Moses J, Iyer R, Illikkal R, Srinivasan S, Aisopos K (2011) Shared resource monitoring and throughput optimization in cloud-computing datacenters. In: IEEE international parallel and distributed processing symposium, pp 1024–1033
34.
Zurück zum Zitat Ge J, Zhang B, Fang Y (2010) Research on the resource monitoring model under cloud computing environment. In: Web information systems and mining, volume 6318 of the series Lecture Notes in Computer Science, pp 111–118 Ge J, Zhang B, Fang Y (2010) Research on the resource monitoring model under cloud computing environment. In: Web information systems and mining, volume 6318 of the series Lecture Notes in Computer Science, pp 111–118
35.
Zurück zum Zitat Wang J, Korambath P, Altintas I, Davis J, Crawl D (2014) Workflow as a service in the cloud: architecture and scheduling algorithms. In: 14th international conference on computational science, vol 29. Elsevier, pp 546–556 Wang J, Korambath P, Altintas I, Davis J, Crawl D (2014) Workflow as a service in the cloud: architecture and scheduling algorithms. In: 14th international conference on computational science, vol 29. Elsevier, pp 546–556
Metadaten
Titel
Proposing an Architecture for Scientific Workflow Management System in Cloud
verfasst von
Vahab Samandi
Debajyoti Mukhopadhyay
Copyright-Jahr
2017
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-3935-5_30

Neuer Inhalt