Skip to main content
Top
Published in: The Journal of Supercomputing 10/2019

11-05-2019

Integer linear programming-based multi-objective scheduling for scientific workflows in multi-cloud environments

Authors: Somayeh Mohammadi, Latif PourKarimi, Hossein Pedram

Published in: The Journal of Supercomputing | Issue 10/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Scientific communities are motivated to schedule the data-intensive scientific workflows in multi-cloud environments, where considerable diverse resources are provided by multiple clouds and resource limitation imposed by individual clouds is overcome. However, this scheduling involves two conflicting objectives: minimizing cost and makespan. In general, dealing with such conflicting criteria is a difficult task. But fortunately recent efficient methods for solving multi-objective optimization problems motivated us to provide a multi-objective model considering minimization of cost and makespan as objectives. For solving this model, we use different scalarization procedures such as weighted-sum, Benson's scalarization and weighted min–max under different scenarios. Moreover, we investigate the stability of obtained solutions and propose a new approach for determining the most stable solution related to weighted-sum and weighted min–max as post-optimality analysis. Results indicate that our proposed weighted-sum approach outperforms the previously developed methods in terms of hypervolume.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Footnotes
3
The Epigenome workflow with 4000 tasks has eight levels. Assume the execution time of each task is in seconds on VM with 1 CCU. The level 5 has 495 tasks with execution time = 9635.01 s. For executing the tasks in level 5 on VMs of Azure with performance 9 CCU, the 56 VMs are needed in which these tasks are executed in three bags with six tasks and 53 bags with nine tasks.
 
5
CloudHarmony Compute Unit.
 
Literature
1.
go back to reference Abdi S, PourKarimi L, Ahmadi M, Zargari F (2018) Cost minimization for bag-of-tasks workflows in a federation of clouds. J Supercomput 74(6):2801–2822CrossRef Abdi S, PourKarimi L, Ahmadi M, Zargari F (2018) Cost minimization for bag-of-tasks workflows in a federation of clouds. J Supercomput 74(6):2801–2822CrossRef
2.
go back to reference Abdi S, PourKarimi L, Ahmadi M, Zargari F (2017) Cost minimization for deadline-constrained bag-of-tasks applications in federated hybrid clouds. Future Gener Comput Syst 71:113–128CrossRef Abdi S, PourKarimi L, Ahmadi M, Zargari F (2017) Cost minimization for deadline-constrained bag-of-tasks applications in federated hybrid clouds. Future Gener Comput Syst 71:113–128CrossRef
3.
go back to reference Nesmachnow S, Iturriaga S, Dorronsoro B (2015) Efficient heuristics for profit optimization of virtual cloud brokers. IEEE Comput Intell Mag 10(1):33–43CrossRef Nesmachnow S, Iturriaga S, Dorronsoro B (2015) Efficient heuristics for profit optimization of virtual cloud brokers. IEEE Comput Intell Mag 10(1):33–43CrossRef
4.
go back to reference Toosi AN, Calheiros RN, Buyya R (2014) Interconnected cloud computing environments: challenges, taxonomy, and survey. ACM Comput Surv (CSUR) 47(1):7CrossRef Toosi AN, Calheiros RN, Buyya R (2014) Interconnected cloud computing environments: challenges, taxonomy, and survey. ACM Comput Surv (CSUR) 47(1):7CrossRef
5.
go back to reference Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 71(9):3373–3418CrossRef Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 71(9):3373–3418CrossRef
6.
go back to reference Schrijver A (1998) Theory of linear and integer programming. Wiley, HobokenMATH Schrijver A (1998) Theory of linear and integer programming. Wiley, HobokenMATH
7.
go back to reference Kadioglu S, Malitsky Y, Sellmann M, Tierney K (2010) ISAC-instance-specific algorithm configuration. In: ECAI, vol 215, pp 751–756 Kadioglu S, Malitsky Y, Sellmann M, Tierney K (2010) ISAC-instance-specific algorithm configuration. In: ECAI, vol 215, pp 751–756
8.
go back to reference Fard HM, Prodan R, Fahringer T (2014) Multi-objective list scheduling of workflow applications in distributed computing infrastructures. J Parallel Distrib Comput 74(3):2152–2165CrossRef Fard HM, Prodan R, Fahringer T (2014) Multi-objective list scheduling of workflow applications in distributed computing infrastructures. J Parallel Distrib Comput 74(3):2152–2165CrossRef
9.
go back to reference Hu H, Li Z, Hu H, Chen J, Ge J, Li C, Chang V (2018) Multi-objective scheduling for scientific workflow in multicloud environment. J Netw Comput Appl 114:108–122CrossRef Hu H, Li Z, Hu H, Chen J, Ge J, Li C, Chang V (2018) Multi-objective scheduling for scientific workflow in multicloud environment. J Netw Comput Appl 114:108–122CrossRef
10.
go back to reference Zhu Z, Zhang G, Li M, Liu X (2016) Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans Parallel Distrib Syst 27(5):1344–1357CrossRef Zhu Z, Zhang G, Li M, Liu X (2016) Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans Parallel Distrib Syst 27(5):1344–1357CrossRef
11.
go back to reference Jena RK (2015) Multi objective task scheduling in cloud environment using nested PSO framework. Proc Comput Sci 57:1219–1227CrossRef Jena RK (2015) Multi objective task scheduling in cloud environment using nested PSO framework. Proc Comput Sci 57:1219–1227CrossRef
12.
go back to reference Wang X, Yeo CS, Buyya R, Su J (2011) Optimizing the makespan and reliability for workflow applications with reputation and a look-ahead genetic algorithm. Future Gener Comput Syst 27(8):1124–1134CrossRef Wang X, Yeo CS, Buyya R, Su J (2011) Optimizing the makespan and reliability for workflow applications with reputation and a look-ahead genetic algorithm. Future Gener Comput Syst 27(8):1124–1134CrossRef
13.
go back to reference Choudhary A, Gupta I, Singh V, Jana PK (2018) A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Future Gener Comput Syst 83:14–26CrossRef Choudhary A, Gupta I, Singh V, Jana PK (2018) A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Future Gener Comput Syst 83:14–26CrossRef
14.
go back to reference Durillo JJ, Prodan R (2014) Multi-objective workflow scheduling in Amazon EC2. Cluster Comput 17(2):169–189CrossRef Durillo JJ, Prodan R (2014) Multi-objective workflow scheduling in Amazon EC2. Cluster Comput 17(2):169–189CrossRef
15.
go back to reference Xu H, Yang B, Qi W, Ahene E (2016) A multi-objective optimization approach to workflow scheduling in clouds considering fault recovery. KSII Trans Internet Inf Syst (TIIS) 10(3):976–995 Xu H, Yang B, Qi W, Ahene E (2016) A multi-objective optimization approach to workflow scheduling in clouds considering fault recovery. KSII Trans Internet Inf Syst (TIIS) 10(3):976–995
16.
go back to reference Qu X, Xiao P, Huang L (2018) Improving the energy efficiency and performance of data-intensive workflows in virtualized clouds. J Supercomput 74(7):2935–2955CrossRef Qu X, Xiao P, Huang L (2018) Improving the energy efficiency and performance of data-intensive workflows in virtualized clouds. J Supercomput 74(7):2935–2955CrossRef
17.
go back to reference Rezaeian A, Naghibzadeh M, Epema DHJ (2019) Fair multiple-workflow scheduling with different quality-of-service goals. J Supercomput 75(2):746–769CrossRef Rezaeian A, Naghibzadeh M, Epema DHJ (2019) Fair multiple-workflow scheduling with different quality-of-service goals. J Supercomput 75(2):746–769CrossRef
18.
go back to reference Poola D, Ramamohanarao K, Buyya R (2014) Fault-tolerant workflow scheduling using spot instances on clouds. Proc Comput Sci 29:523–533CrossRef Poola D, Ramamohanarao K, Buyya R (2014) Fault-tolerant workflow scheduling using spot instances on clouds. Proc Comput Sci 29:523–533CrossRef
19.
go back to reference de Oliveira D, Ocaña KACS, Baião F, Mattoso M (2012) A provenance-based adaptive scheduling heuristic for parallel scientific workflows in clouds. J Grid Comput 10(3):521–552CrossRef de Oliveira D, Ocaña KACS, Baião F, Mattoso M (2012) A provenance-based adaptive scheduling heuristic for parallel scientific workflows in clouds. J Grid Comput 10(3):521–552CrossRef
21.
go back to reference Pietri I, Malawski M, Juve G, Deelman E, Nabrzyski J, Sakellariou R (2013) Energy-constrained provisioning for scientific workflow ensembles. In: Third International Conference on Cloud and Green Computing (CGC), 2013. IEEE, pp 34–41 Pietri I, Malawski M, Juve G, Deelman E, Nabrzyski J, Sakellariou R (2013) Energy-constrained provisioning for scientific workflow ensembles. In: Third International Conference on Cloud and Green Computing (CGC), 2013. IEEE, pp 34–41
22.
go back to reference Zeng L, Veeravalli B, Li X (2015) SABA: a security-aware and budget-aware workflow scheduling strategy in clouds. J Parallel Distrib Comput 75:141–151CrossRef Zeng L, Veeravalli B, Li X (2015) SABA: a security-aware and budget-aware workflow scheduling strategy in clouds. J Parallel Distrib Comput 75:141–151CrossRef
23.
go back to reference Mohammadi S, Pedram H, PourKarimi L (2018) Integer linear programming-based cost optimization for scheduling scientific workflows in multi-cloud environments. J Supercomput 74(9):4717–4745CrossRef Mohammadi S, Pedram H, PourKarimi L (2018) Integer linear programming-based cost optimization for scheduling scientific workflows in multi-cloud environments. J Supercomput 74(9):4717–4745CrossRef
24.
go back to reference Rehman A, Hussain SS, ur Rehman Z, Zia S, Shamshirband S (2018) Multi-objective approach of energy efficient workflow scheduling in cloud environments. Concurr Comput Pract Exp 31(8):e4949CrossRef Rehman A, Hussain SS, ur Rehman Z, Zia S, Shamshirband S (2018) Multi-objective approach of energy efficient workflow scheduling in cloud environments. Concurr Comput Pract Exp 31(8):e4949CrossRef
25.
go back to reference Arabnejad V, Bubendorfer K, Ng B (2019) Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1):29–44CrossRef Arabnejad V, Bubendorfer K, Ng B (2019) Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1):29–44CrossRef
26.
go back to reference Bharathi S, Chervenak A, Deelman E, Mehta G, Su M-H, Vahi K (2008) Characterization of scientific workflows. In: Third Workshop on workflows in support of large-scale science, 2008. WORKS 2008. IEEE, pp 1–10 Bharathi S, Chervenak A, Deelman E, Mehta G, Su M-H, Vahi K (2008) Characterization of scientific workflows. In: Third Workshop on workflows in support of large-scale science, 2008. WORKS 2008. IEEE, pp 1–10
27.
go back to reference Ehrgott M (2005) Multicriteria optimization, vol 491. Springer, BerlinMATH Ehrgott M (2005) Multicriteria optimization, vol 491. Springer, BerlinMATH
28.
go back to reference Benson HP (1978) Existence of efficient solutions for vector maximization problems. J Optim Theory Appl 26(4):569–580MathSciNetCrossRef Benson HP (1978) Existence of efficient solutions for vector maximization problems. J Optim Theory Appl 26(4):569–580MathSciNetCrossRef
29.
go back to reference IBMI (2009) CPLEX, V12. 1: user’s manual for CPLEX. International Business Machines Corporation 46(53):157 IBMI (2009) CPLEX, V12. 1: user’s manual for CPLEX. International Business Machines Corporation 46(53):157
30.
go back to reference Topcuoglu H, Hariri S, Wu M-Y (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274CrossRef Topcuoglu H, Hariri S, Wu M-Y (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274CrossRef
31.
go back to reference Zitzler E, Thiele L, Laumanns M, Fonseca CM, Da Fonseca VG (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans Evol Comput 7(2):117–132CrossRef Zitzler E, Thiele L, Laumanns M, Fonseca CM, Da Fonseca VG (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans Evol Comput 7(2):117–132CrossRef
32.
go back to reference Bradstreet L (2011) The hypervolume indicator for multi-objective optimisation: calculation and use. University of Western Australia Bradstreet L (2011) The hypervolume indicator for multi-objective optimisation: calculation and use. University of Western Australia
33.
go back to reference Sitarz S (2012) Mean value and volume-based sensitivity analysis for Olympic rankings. Eur J Oper Res 216(1):232–238MathSciNetCrossRef Sitarz S (2012) Mean value and volume-based sensitivity analysis for Olympic rankings. Eur J Oper Res 216(1):232–238MathSciNetCrossRef
Metadata
Title
Integer linear programming-based multi-objective scheduling for scientific workflows in multi-cloud environments
Authors
Somayeh Mohammadi
Latif PourKarimi
Hossein Pedram
Publication date
11-05-2019
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 10/2019
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-019-02877-8

Other articles of this Issue 10/2019

The Journal of Supercomputing 10/2019 Go to the issue

Premium Partner