Skip to main content

2018 | OriginalPaper | Buchkapitel

5. Resource Provisioning Strategy for Scientific Workflows in Cloud Computing Environment

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Cloud computing has emerged as a computing paradigm to solve large-scale problems. The main intent of Cloud computing is to provide inexpensive computing resources on a pay-as-you-go basis, which is promptly gaining momentum as a substitute for traditional information technology (IT)-based organizations. Therefore, the increased utilization of Clouds makes successful execution of scientific applications a vital research area. As more and more users have started to store and process their real-time data in Cloud environments, resource provisioning and scheduling of huge Data processing jobs becomes a key element of consideration for efficient execution of scientific applications. The base of any real-time system is a resource, and to manage the resources to handle workflow applications in Cloud computing environment is a very tedious task. An inefficient resource management system can have a direct negative effect on performance and cost and indirect effect on functionality of the system. Indeed, some functions provided by the system may become too expensive or may be avoided due to poor performance. Thus, Cloud computing faces the challenge of resource management, especially with respect to choosing resource provisioning strategies and suitable algorithms for particular applications. The major components of resource management systems are resource provisioning and scheduling. If any system is able to fulfill the requirements of these two components, the execution of scientific workflow applications will become much easier. This chapter discusses the fundamental concepts supporting Cloud computing and resource management system terms and the relationship between them. It reflects the essential perceptions behind the Cloud resource provisioning strategies. The chapter also identifies requirements based on user’s applications associated with handling real-time data. A model for resource provisioning based on user’s requirements to maximize efficiency and analysis of scientific workflows is also proposed. QoS parameter (s) based resource provisioning strategy has been proposed for workflow applications in cloud computing environment. Validation of resource provisioning strategies is presented in this book chapter.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat R. Abinaya, P. Harris, A Novel Resource Provisioning Approach for Virtualized Environment (2016) R. Abinaya, P. Harris, A Novel Resource Provisioning Approach for Virtualized Environment (2016)
3.
Zurück zum Zitat M. Armbrust, A. Fox, R. Griffith, A.D. Joseph, R.H. Katz, A. Konwinski, G. Lee, D.A. Patterson, A. Rabkin, I. Stoica et al., Above the clouds: a Berkeley view of cloud computing (2009) M. Armbrust, A. Fox, R. Griffith, A.D. Joseph, R.H. Katz, A. Konwinski, G. Lee, D.A. Patterson, A. Rabkin, I. Stoica et al., Above the clouds: a Berkeley view of cloud computing (2009)
6.
Zurück zum Zitat E. Burke, G. Kendall, D.L. Silva, R. OBrien, E. Soubeiga, An ant algorithm hyperheuristic for the project presentation scheduling problem, in 2005 IEEE Congress on Evolutionary Computation, vol. 3 (IEEE, 2005), pp. 2263–2270 E. Burke, G. Kendall, D.L. Silva, R. OBrien, E. Soubeiga, An ant algorithm hyperheuristic for the project presentation scheduling problem, in 2005 IEEE Congress on Evolutionary Computation, vol. 3 (IEEE, 2005), pp. 2263–2270
7.
Zurück zum Zitat E.K. Burke, M. Gendreau, G. Hyde, G. Kendall, G. Ochoa, E. O zcan, R. Qu, Hyperheuristics: a survey of the state of the art. J. Oper. Res. Soc. 64(12), 1695–1724 (2013) E.K. Burke, M. Gendreau, G. Hyde, G. Kendall, G. Ochoa, E. O zcan, R. Qu, Hyperheuristics: a survey of the state of the art. J. Oper. Res. Soc. 64(12), 1695–1724 (2013)
8.
Zurück zum Zitat R. Buyya, C.S. Yeo, S. Venugopal, J. Broberg, I. Brandic, Cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6), 599–616 (2009)CrossRef R. Buyya, C.S. Yeo, S. Venugopal, J. Broberg, I. Brandic, Cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6), 599–616 (2009)CrossRef
9.
Zurück zum Zitat R.N. Calheiros, R. Buyya, Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)CrossRef R.N. Calheiros, R. Buyya, Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)CrossRef
10.
Zurück zum Zitat R.N. Calheiros, R. Ranjan, R. Buyya, Virtual machine provisioning based on analytical performance and qos in cloud computing environments, in 2011 International Conference on Parallel Processing (ICPP) (IEEE, 2011), pp. 295–304 R.N. Calheiros, R. Ranjan, R. Buyya, Virtual machine provisioning based on analytical performance and qos in cloud computing environments, in 2011 International Conference on Parallel Processing (ICPP) (IEEE, 2011), pp. 295–304
11.
Zurück zum Zitat S. Chaisiri, B.S. Lee, D. Niyato, Optimization of resource provisioning cost in cloud computing. IEEE Trans. Serv. Comput. 5(2), 164–177 (2012)CrossRef S. Chaisiri, B.S. Lee, D. Niyato, Optimization of resource provisioning cost in cloud computing. IEEE Trans. Serv. Comput. 5(2), 164–177 (2012)CrossRef
12.
Zurück zum Zitat P. Cowling, G. Kendall, L. Han, An investigation of a hyperheuristic genetic algorithm applied to a trainer scheduling problem, in Proceedings of the 2002 Congress on Evolutionary Computation, 2002. CEC02, vol. 2 (IEEE, 2002), pp. 1185–1190 P. Cowling, G. Kendall, L. Han, An investigation of a hyperheuristic genetic algorithm applied to a trainer scheduling problem, in Proceedings of the 2002 Congress on Evolutionary Computation, 2002. CEC02, vol. 2 (IEEE, 2002), pp. 1185–1190
13.
Zurück zum Zitat D.G. Feitelson, Workload Modeling for Computer Systems Performance evaluation (Cambridge University Press, Cambridge, 2015)CrossRef D.G. Feitelson, Workload Modeling for Computer Systems Performance evaluation (Cambridge University Press, Cambridge, 2015)CrossRef
14.
Zurück zum Zitat I. Foster, Y. Zhao, I. Raicu, S. Lu, Cloud computing and grid computing 360-degree compared, in Grid Computing Environments Workshop, 2008. GCE08 (IEEE, 2008), p. 110 I. Foster, Y. Zhao, I. Raicu, S. Lu, Cloud computing and grid computing 360-degree compared, in Grid Computing Environments Workshop, 2008. GCE08 (IEEE, 2008), p. 110
15.
Zurück zum Zitat A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, Above the clouds: a Berkeley view of cloud computing. Dept. Electr. Eng. Comput. Sci. Univ. California Berkeley, Rep. UCB/EECS 28(13), 2009 (2009) A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, Above the clouds: a Berkeley view of cloud computing. Dept. Electr. Eng. Comput. Sci. Univ. California Berkeley, Rep. UCB/EECS 28(13), 2009 (2009)
16.
Zurück zum Zitat M. Frincu, S. Genaud, J. Gossa, Comparing provisioning and scheduling strategies for workflows on clouds, in Workshop Proceedings of 28th IEEE International Parallel and Distributed Processing Symposium (IEEE, 2013), pp. 2101–2110 M. Frincu, S. Genaud, J. Gossa, Comparing provisioning and scheduling strategies for workflows on clouds, in Workshop Proceedings of 28th IEEE International Parallel and Distributed Processing Symposium (IEEE, 2013), pp. 2101–2110
17.
Zurück zum Zitat S. Genaud, J. Gossa, Cost-wait trade-offs in client-side resource provisioning with elastic clouds, in 4th IEEE International Conference on Cloud Computing (CLOUD 2011) (IEEE, 2011) S. Genaud, J. Gossa, Cost-wait trade-offs in client-side resource provisioning with elastic clouds, in 4th IEEE International Conference on Cloud Computing (CLOUD 2011) (IEEE, 2011)
18.
Zurück zum Zitat F. Glover, Tabu search-part i. ORSA J. Comput. 1(3), 190–206 (1989)CrossRef F. Glover, Tabu search-part i. ORSA J. Comput. 1(3), 190–206 (1989)CrossRef
19.
Zurück zum Zitat M.A. Iverson, F. O zguner, G.J. Follen, Parallelizing existing applications in a distributed heterogeneous environment, in 4TH Heterogeneous Computing Workshop HCW95 (Citeseer, 1995) M.A. Iverson, F. O zguner, G.J. Follen, Parallelizing existing applications in a distributed heterogeneous environment, in 4TH Heterogeneous Computing Workshop HCW95 (Citeseer, 1995)
20.
Zurück zum Zitat B. Jennings, R. Stadler, Resource management in clouds: survey and research challenges. J. Netw. Syst. Manag. 23(3), 567–619 (2015)CrossRef B. Jennings, R. Stadler, Resource management in clouds: survey and research challenges. J. Netw. Syst. Manag. 23(3), 567–619 (2015)CrossRef
21.
Zurück zum Zitat G. Juve, E. Deelman, G.B. Berriman, B.P. Berman, P. Maechling, An evaluation of the cost and performance of scientific workflows on amazon ec2. J. Grid Comput. 10(1), 521 (2012)CrossRef G. Juve, E. Deelman, G.B. Berriman, B.P. Berman, P. Maechling, An evaluation of the cost and performance of scientific workflows on amazon ec2. J. Grid Comput. 10(1), 521 (2012)CrossRef
22.
Zurück zum Zitat G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B.P. Berman, P. Maechling, Scientific workflow applications on amazon ec2, in 2009 5th IEEE International Conference on E-Science Workshops (IEEE, 2009), pp. 59–66 G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B.P. Berman, P. Maechling, Scientific workflow applications on amazon ec2, in 2009 5th IEEE International Conference on E-Science Workshops (IEEE, 2009), pp. 59–66
23.
Zurück zum Zitat G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B.P. Berman, P. Maechling, Data sharing options for scientific workflows on amazon ec2, in Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (IEEE Computer Society, 2010), p. 19 G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B.P. Berman, P. Maechling, Data sharing options for scientific workflows on amazon ec2, in Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (IEEE Computer Society, 2010), p. 19
24.
Zurück zum Zitat C. Lin, S. Lu, Scheduling scientific workflows elastically for cloud computing, in 2011 IEEE International Conference on Cloud Computing (CLOUD) (IEEE, 2011), pp. 746–747 C. Lin, S. Lu, Scheduling scientific workflows elastically for cloud computing, in 2011 IEEE International Conference on Cloud Computing (CLOUD) (IEEE, 2011), pp. 746–747
25.
Zurück zum Zitat W. Lin, C. Liang, J.Z. Wang, R. Buyya, Bandwidth-aware divisible task scheduling for cloud computing. Softw. Pract. Exper. 44(2), 163–174 (2014)CrossRef W. Lin, C. Liang, J.Z. Wang, R. Buyya, Bandwidth-aware divisible task scheduling for cloud computing. Softw. Pract. Exper. 44(2), 163–174 (2014)CrossRef
26.
Zurück zum Zitat M. Malawski, G. Juve, E. Deelman, J. Nabrzyski, Cost-and deadline-constrained provisioning for scientific workflow ensembles in IAAS clouds, in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (IEEE Computer Society Press, 2012), p. 22 M. Malawski, G. Juve, E. Deelman, J. Nabrzyski, Cost-and deadline-constrained provisioning for scientific workflow ensembles in IAAS clouds, in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (IEEE Computer Society Press, 2012), p. 22
27.
Zurück zum Zitat E. Michon, J. Gossa, S. Genaud, Free elasticity and free CPU power for scientific workloads on IaaS Clouds, in 18th IEEE International Conference on Parallel and Distributed Systems (IEEE, Singapour, Singapore, 2012), http://hal.inria.fr/hal-00733155 E. Michon, J. Gossa, S. Genaud, Free elasticity and free CPU power for scientific workloads on IaaS Clouds, in 18th IEEE International Conference on Parallel and Distributed Systems (IEEE, Singapour, Singapore, 2012), http://​hal.​inria.​fr/​hal-00733155
28.
Zurück zum Zitat I. Chana, Rajni, Bacterial foraging based hyper-heuristic for resource scheduling in grid computing. Future Gener. Comput. Syst. 29(3), 751–762 (2013) I. Chana, Rajni, Bacterial foraging based hyper-heuristic for resource scheduling in grid computing. Future Gener. Comput. Syst. 29(3), 751–762 (2013)
29.
Zurück zum Zitat A. Rajni, An empirical study of vm provisioning strategies on IAAS cloud, in 2016 IEEE 18th International Conference on High Performance Computing and Communications (IEEE, 2016) A. Rajni, An empirical study of vm provisioning strategies on IAAS cloud, in 2016 IEEE 18th International Conference on High Performance Computing and Communications (IEEE, 2016)
30.
Zurück zum Zitat J. Sen, Security and privacy issues in cloud computing, in Architectures and Protocols for Secure Information Technology Infrastructures (2013), p. 145 J. Sen, Security and privacy issues in cloud computing, in Architectures and Protocols for Secure Information Technology Infrastructures (2013), p. 145
31.
Zurück zum Zitat J. Shi, J. Luo, F. Dong, J. Zhang, J. Zhang, Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints. Cluster Comput. 19(1), 167182 (2016)CrossRef J. Shi, J. Luo, F. Dong, J. Zhang, J. Zhang, Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints. Cluster Comput. 19(1), 167182 (2016)CrossRef
32.
Zurück zum Zitat S. Srinivasan, G. Juve, R.F. Da Silva, K. Vahi, E. Deelman, A cleanup algorithm for implementing storage constraints in scientific workflow executions, in 2014 9th Workshop on Workflows in Support of Large-Scale Science (WORKS) (IEEE, 2014), pp. 41–49 S. Srinivasan, G. Juve, R.F. Da Silva, K. Vahi, E. Deelman, A cleanup algorithm for implementing storage constraints in scientific workflow executions, in 2014 9th Workshop on Workflows in Support of Large-Scale Science (WORKS) (IEEE, 2014), pp. 41–49
33.
Zurück zum Zitat C. Szabo, Q.Z. Sheng, T. Kroeger, Y. Zhang, J. Yu, Science in the cloud: Allocation and execution of data-intensive scientific workflows. J. Grid Comput. 120 (2013) C. Szabo, Q.Z. Sheng, T. Kroeger, Y. Zhang, J. Yu, Science in the cloud: Allocation and execution of data-intensive scientific workflows. J. Grid Comput. 120 (2013)
34.
Zurück zum Zitat D. Villegas, A. Antoniou, S.M. Sadjadi, A. Iosup, An analysis of provisioning and allocation policies for infrastructure-as-a-service clouds, in 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) (IEEE, 2012), pp. 612–619 D. Villegas, A. Antoniou, S.M. Sadjadi, A. Iosup, An analysis of provisioning and allocation policies for infrastructure-as-a-service clouds, in 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) (IEEE, 2012), pp. 612–619
35.
Zurück zum Zitat Z. Wu, X. Liu, Z. Ni, D. Yuan, Y. Yang, A market-oriented hierarchical scheduling strategy in cloud workflow systems. J. Supercomput. 63(1), 256–293 (2013)CrossRef Z. Wu, X. Liu, Z. Ni, D. Yuan, Y. Yang, A market-oriented hierarchical scheduling strategy in cloud workflow systems. J. Supercomput. 63(1), 256–293 (2013)CrossRef
36.
Zurück zum Zitat A. Zhou, B. He, C. Liu, Monetary cost optimizations for hosting workflow-as-a-service in IAAS clouds (2013) A. Zhou, B. He, C. Liu, Monetary cost optimizations for hosting workflow-as-a-service in IAAS clouds (2013)
Metadaten
Titel
Resource Provisioning Strategy for Scientific Workflows in Cloud Computing Environment
verfasst von
Rajni Aron
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-73676-1_5

Premium Partner