Skip to main content
Erschienen in: The Journal of Supercomputing 8/2020

26.05.2020

A cost saving and load balancing task scheduling model for computational biology in heterogeneous cloud datacenters

verfasst von: Wenwei Cai, Jiaxian Zhu, Weihua Bai, Weiwei Lin, Naqin Zhou, Keqin Li

Erschienen in: The Journal of Supercomputing | Ausgabe 8/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Cloud-based scientific workflow systems can play an important role in the development of cost-effective bioinformatics analysis applications. There are differences in the cost control and performance of many kinds of servers in heterogeneous cloud data centers for bioinformatics workflows running, which can lead to imbalance between operational/maintenance management costs and quality of service of server clusters. A task scheduling model that responds to the peaks and valleys of task sequencing—the number of tasks that arrive in a given unit of time—is related to indicators such as cost saving, load balancing and system performance (average task wait time, average response time and throughput). This study proposes a large-scale cost-saving and load-balancing scheduling model, called HDCBS, for the optimization of system throughput. First, queuing theory is used to model each computing node as an independent queuing system and to obtain the average system wait time and average task response time. Then, using convex optimization theory, a task assignment solution is proposed with a load-balancing mechanism. The validity of the task scheduling model is verified by simulation experiments, and the model performance is further validated through a comparison with other frequently used scheduling methods. The simulation results show that the credibility of HDCBS is greater than 95% in task scheduling.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Lu C, Ye K, Xu G, Xu C-Z, Bai T (2017) Imbalance in the cloud: an analysis on alibaba cluster trace. In: 2017 IEEE International Conference on Big Data (Big Data), IEEE, pp 2884–2892 Lu C, Ye K, Xu G, Xu C-Z, Bai T (2017) Imbalance in the cloud: an analysis on alibaba cluster trace. In: 2017 IEEE International Conference on Big Data (Big Data), IEEE, pp 2884–2892
2.
Zurück zum Zitat Cheng Y, Chai Z, Anwar A (2018) Characterizing co-located datacenter workloads: an alibaba case study. In: Proceedings of the 9th Asia-Pacific Workshop on Systems, APSys 2018, Jeju Island, Republic of Korea, pp 12:1–12:3 Cheng Y, Chai Z, Anwar A (2018) Characterizing co-located datacenter workloads: an alibaba case study. In: Proceedings of the 9th Asia-Pacific Workshop on Systems, APSys 2018, Jeju Island, Republic of Korea, pp 12:1–12:3
3.
Zurück zum Zitat Jiang Congfeng, Han Guangjie, Lin Jiangbin, Jia Gangyong, Shi Weisong, Wan Jian (2019) Characteristics of co-allocated online services and batch jobs in internet data centers: a case study from alibaba cloud. IEEE Access 7:22495–22508 Jiang Congfeng, Han Guangjie, Lin Jiangbin, Jia Gangyong, Shi Weisong, Wan Jian (2019) Characteristics of co-allocated online services and batch jobs in internet data centers: a case study from alibaba cloud. IEEE Access 7:22495–22508
4.
Zurück zum Zitat Kameda H, Li J, Kim C, Zhang Y (2012) Optimal load balancing in distributed computer systems. Springer, New YorkMATH Kameda H, Li J, Kim C, Zhang Y (2012) Optimal load balancing in distributed computer systems. Springer, New YorkMATH
5.
Zurück zum Zitat Domanal SG, Reddy GRM (2014) Optimal load balancing in cloud computing by efficient utilization of virtual machines. In: Sixth International Conference on Communication Systems and Networks, COMSNETS 2014, Bangalore, India, pp 1–4 Domanal SG, Reddy GRM (2014) Optimal load balancing in cloud computing by efficient utilization of virtual machines. In: Sixth International Conference on Communication Systems and Networks, COMSNETS 2014, Bangalore, India, pp 1–4
6.
Zurück zum Zitat Andrews Jeffrey G, Singh Sarabjot, Ye Qiaoyang, Lin Xingqin, Dhillon Harpreet S (2014) An overview of load balancing in hetnets: old myths and open problems. IEEE Wireless Commun 21(2):18–25 Andrews Jeffrey G, Singh Sarabjot, Ye Qiaoyang, Lin Xingqin, Dhillon Harpreet S (2014) An overview of load balancing in hetnets: old myths and open problems. IEEE Wireless Commun 21(2):18–25
7.
Zurück zum Zitat Lin Weiwei, Siyao Xu, He Ligang, Li Jin (2017) Multi-resource scheduling and power simulation for cloud computing. Inf Sci 397:168–186 Lin Weiwei, Siyao Xu, He Ligang, Li Jin (2017) Multi-resource scheduling and power simulation for cloud computing. Inf Sci 397:168–186
8.
Zurück zum Zitat Hondo F, Wercelens P, da Silva WMC, Castro K, Santana I, Walter MET, de Araújo APF, Holanda M, Lifschitz S (2017) Data provenance management for bioinformatics workflows using NOSQL database systems in a cloud computing environment. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2017, Kansas City, MO, USA, pp 1929–1934 Hondo F, Wercelens P, da Silva WMC, Castro K, Santana I, Walter MET, de Araújo APF, Holanda M, Lifschitz S (2017) Data provenance management for bioinformatics workflows using NOSQL database systems in a cloud computing environment. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2017, Kansas City, MO, USA, pp 1929–1934
9.
Zurück zum Zitat Liu Bo, Madduri Ravi K, Sotomayor Borja, Chard Kyle, Lacinski Lukasz, Dave Utpal J, Li Jianqiang, Liu Chunchen, Foster Ian T (2014) Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses. J Biomed Inf 49:119–133 Liu Bo, Madduri Ravi K, Sotomayor Borja, Chard Kyle, Lacinski Lukasz, Dave Utpal J, Li Jianqiang, Liu Chunchen, Foster Ian T (2014) Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses. J Biomed Inf 49:119–133
10.
Zurück zum Zitat Abouelhoda Mohamed, Issa Shadi, Ghanem Moustafa (2013) Towards scalable and cost-aware bioinformatics workflow execution in the cloud—recent advances to the tavaxy workflow system. Fundam Inf 128(3):255–280 Abouelhoda Mohamed, Issa Shadi, Ghanem Moustafa (2013) Towards scalable and cost-aware bioinformatics workflow execution in the cloud—recent advances to the tavaxy workflow system. Fundam Inf 128(3):255–280
11.
Zurück zum Zitat Emeakaroha Vincent C, Maurer Michael, Stern Patrick, Labaj Pawel P, Brandic Ivona, Kreil David P (2013) Managing and optimizing bioinformatics workflows for data analysis in clouds. J Grid Comput 11(3):407–428 Emeakaroha Vincent C, Maurer Michael, Stern Patrick, Labaj Pawel P, Brandic Ivona, Kreil David P (2013) Managing and optimizing bioinformatics workflows for data analysis in clouds. J Grid Comput 11(3):407–428
12.
Zurück zum Zitat Xie Z, Han L, Baldock RA (2013) Augmented petri net cost model for optimisation of large bioinformatics workflows using cloud. In: Seventh UKSim/AMSS European Modelling Symposium, EMS 2013, Manchester UK, pp 201–205 Xie Z, Han L, Baldock RA (2013) Augmented petri net cost model for optimisation of large bioinformatics workflows using cloud. In: Seventh UKSim/AMSS European Modelling Symposium, EMS 2013, Manchester UK, pp 201–205
13.
Zurück zum Zitat Bai W-H, Xi J-Q, Zhu J-X, Huang S-W (2015) Performance analysis of heterogeneous data centers in cloud computing using a complex queuing model. In: Mathematical Problems in Engineering 2015 Bai W-H, Xi J-Q, Zhu J-X, Huang S-W (2015) Performance analysis of heterogeneous data centers in cloud computing using a complex queuing model. In: Mathematical Problems in Engineering 2015
14.
Zurück zum Zitat Jin Y, Gao Y, Qian Z, Zhai M, Peng H, Lu S (2016) Workload-aware scheduling across geo-distributed data centers. In: 2016 IEEE Trustcom/BigDataSE/ISPA, Tianjin, China, pp 1455–1462 Jin Y, Gao Y, Qian Z, Zhai M, Peng H, Lu S (2016) Workload-aware scheduling across geo-distributed data centers. In: 2016 IEEE Trustcom/BigDataSE/ISPA, Tianjin, China, pp 1455–1462
15.
Zurück zum Zitat Chen Shang-Liang, Chen Yun-Yao, Kuo Suang-Hong (2017) CLB: a novel load balancing architecture and algorithm for cloud services. Comput Electr Eng 58:154–160 Chen Shang-Liang, Chen Yun-Yao, Kuo Suang-Hong (2017) CLB: a novel load balancing architecture and algorithm for cloud services. Comput Electr Eng 58:154–160
16.
Zurück zum Zitat Tripathi R, Vignesh S, Tamarapalli V, Chronopoulos AT, Siar H (2017) Non-cooperative power and latency aware load balancing in distributed data centers. J Parallel Distrib Comput 107:76–86 Tripathi R, Vignesh S, Tamarapalli V, Chronopoulos AT, Siar H (2017) Non-cooperative power and latency aware load balancing in distributed data centers. J Parallel Distrib Comput 107:76–86
17.
Zurück zum Zitat Panda Sanjaya K, Jana Prasanta K (2018) Normalization-based task scheduling algorithms for heterogeneous multi-cloud environment. Inf Syst Front 20(2):373–399 Panda Sanjaya K, Jana Prasanta K (2018) Normalization-based task scheduling algorithms for heterogeneous multi-cloud environment. Inf Syst Front 20(2):373–399
18.
Zurück zum Zitat Cao Junwei, Hwang Kai, Li Keqin, Zomaya Albert Y (2013) Optimal multiserver configuration for profit maximization in cloud computing. IEEE Trans Parallel Distrib Syst 24(6):1087–1096 Cao Junwei, Hwang Kai, Li Keqin, Zomaya Albert Y (2013) Optimal multiserver configuration for profit maximization in cloud computing. IEEE Trans Parallel Distrib Syst 24(6):1087–1096
19.
Zurück zum Zitat Chiang Y-J, Ouyang Y-C (2014) Profit optimization in SLA-aware cloud services with a finite capacity queuing model. In: Mathematical Problems in Engineering 2014 Chiang Y-J, Ouyang Y-C (2014) Profit optimization in SLA-aware cloud services with a finite capacity queuing model. In: Mathematical Problems in Engineering 2014
20.
Zurück zum Zitat Cao J, Li K, Stojmenovic I (2014) Optimal power allocation and load distribution for multiple heterogeneous multicore server processors across clouds and data centers. IEEE Trans Comput 63(1):45–58MathSciNetMATH Cao J, Li K, Stojmenovic I (2014) Optimal power allocation and load distribution for multiple heterogeneous multicore server processors across clouds and data centers. IEEE Trans Comput 63(1):45–58MathSciNetMATH
21.
Zurück zum Zitat Yuan H, Bi J, Zhou M (2019) Multi-queue scheduling of heterogeneous tasks with bounded response time in hybrid green IAAS clouds. IEEE Trans Ind Inf 15(10):5404–5412 Yuan H, Bi J, Zhou M (2019) Multi-queue scheduling of heterogeneous tasks with bounded response time in hybrid green IAAS clouds. IEEE Trans Ind Inf 15(10):5404–5412
22.
Zurück zum Zitat Gnimpieba EZ, Thavappiragasam M, Chango A, Conn B, Lushbough CM (2015) Sbmldock: Docker driven systems biology tool development and usage. In: International Conference on Computational Methods in Systems Biology. Springer, New York, pp 282–285 Gnimpieba EZ, Thavappiragasam M, Chango A, Conn B, Lushbough CM (2015) Sbmldock: Docker driven systems biology tool development and usage. In: International Conference on Computational Methods in Systems Biology. Springer, New York, pp 282–285
23.
Zurück zum Zitat Leggett RM, Heavens D, Caccamo M, Clark MD, Davey RP (2016) Nanook: multi-reference alignment analysis of nanopore sequencing data, quality and error profiles. Bioinformatics 32(1):142–144 Leggett RM, Heavens D, Caccamo M, Clark MD, Davey RP (2016) Nanook: multi-reference alignment analysis of nanopore sequencing data, quality and error profiles. Bioinformatics 32(1):142–144
24.
Zurück zum Zitat Ondov BD, Treangen TJ, Melsted P, Mallonee AB, Bergman NH, Koren S, Phillippy AM (2016) Mash: fast genome and metagenome distance estimation using minhash. Genom Biol 17(1):132 Ondov BD, Treangen TJ, Melsted P, Mallonee AB, Bergman NH, Koren S, Phillippy AM (2016) Mash: fast genome and metagenome distance estimation using minhash. Genom Biol 17(1):132
25.
Zurück zum Zitat Liu Q, Yu Z (2018) The elasticity and plasticity in semi-containerized co-locating cloud workload: a view from alibaba trace. In: Proceedings of the ACM Symposium on Cloud Computing, SoCC 2018, Carlsbad, CA, USA, pp 347–360 Liu Q, Yu Z (2018) The elasticity and plasticity in semi-containerized co-locating cloud workload: a view from alibaba trace. In: Proceedings of the ACM Symposium on Cloud Computing, SoCC 2018, Carlsbad, CA, USA, pp 347–360
26.
Zurück zum Zitat Alam M, Shakil KA, Sethi S (2016) Analysis and clustering of workload in Google cluster trace based on resource usage. In 2016 IEEE International Conference on Computational Science and Engineering, CSE 2016, and IEEE International Conference on Embedded and Ubiquitous Computing, EUC 2016, and 15th International Symposium on Distributed Computing and Applications for Business Engineering, DCABES 2016, Paris, France, pp 740–747 Alam M, Shakil KA, Sethi S (2016) Analysis and clustering of workload in Google cluster trace based on resource usage. In 2016 IEEE International Conference on Computational Science and Engineering, CSE 2016, and IEEE International Conference on Embedded and Ubiquitous Computing, EUC 2016, and 15th International Symposium on Distributed Computing and Applications for Business Engineering, DCABES 2016, Paris, France, pp 740–747
27.
Zurück zum Zitat Shortle JF, Thompson JM, Gross D, Harris CM (2018) Fundamentals of queueing theory, vol 399. Wiley, HobokenMATH Shortle JF, Thompson JM, Gross D, Harris CM (2018) Fundamentals of queueing theory, vol 399. Wiley, HobokenMATH
28.
Zurück zum Zitat Boyd Stephen, Vandenberghe Lieven (2004) Convex optimization. Cambridge University Press, CambridgeMATH Boyd Stephen, Vandenberghe Lieven (2004) Convex optimization. Cambridge University Press, CambridgeMATH
29.
Zurück zum Zitat Ren Xiaoqi, Ananthanarayanan Ganesh, Wierman Adam, Minlan Yu (2015) Hopper: decentralized speculation-aware cluster scheduling at scale. Comput Commun Rev 45(5):379–392 Ren Xiaoqi, Ananthanarayanan Ganesh, Wierman Adam, Minlan Yu (2015) Hopper: decentralized speculation-aware cluster scheduling at scale. Comput Commun Rev 45(5):379–392
30.
Zurück zum Zitat Margolies Robert, Sridharan Ashwin, Aggarwal Vaneet, Jana Rittwik, Shankaranarayanan N K, Vaishampayan Vinay A, Zussman Gil (2016) Exploiting mobility in proportional fair cellular scheduling: measurements and algorithms. IEEE/ACM Trans Netw 24(1):355–367 Margolies Robert, Sridharan Ashwin, Aggarwal Vaneet, Jana Rittwik, Shankaranarayanan N K, Vaishampayan Vinay A, Zussman Gil (2016) Exploiting mobility in proportional fair cellular scheduling: measurements and algorithms. IEEE/ACM Trans Netw 24(1):355–367
31.
Zurück zum Zitat Singh Sarabjot, Geraseminko Mikhail, Yeh Shu-ping, Himayat Nageen, Talwar Shilpa (2016) Proportional fair traffic splitting and aggregation in heterogeneous wireless networks. IEEE Commun Lett 20(5):1010–1013 Singh Sarabjot, Geraseminko Mikhail, Yeh Shu-ping, Himayat Nageen, Talwar Shilpa (2016) Proportional fair traffic splitting and aggregation in heterogeneous wireless networks. IEEE Commun Lett 20(5):1010–1013
32.
Zurück zum Zitat Cai Weihong, Yang Junjie, Yidan Yu, Song Youyi, Zhou Teng, Qin Jing (2020) Pso-elm: a hybrid learning model for short-term traffic flow forecasting. IEEE Access 8:6505–6514 Cai Weihong, Yang Junjie, Yidan Yu, Song Youyi, Zhou Teng, Qin Jing (2020) Pso-elm: a hybrid learning model for short-term traffic flow forecasting. IEEE Access 8:6505–6514
33.
Zurück zum Zitat Cai L, Yu Y, Zhang S, Song Y, Xiong Z, Zhou T (2020) A sample-rebalanced outlier-rejected k-nearest neighbour regression model for short-term traffic flow forecasting. IEEE Access 1–11 Cai L, Yu Y, Zhang S, Song Y, Xiong Z, Zhou T (2020) A sample-rebalanced outlier-rejected k-nearest neighbour regression model for short-term traffic flow forecasting. IEEE Access 1–11
34.
Zurück zum Zitat Cai Lingru, Lei Mingqin, Zhang Shuangyi, Yidan Yu, Zhou Teng, Qin Jing (2020) A noise-immune lstm network for short-term traffic flow forecasting. Chaos 30(3):1–10 Cai Lingru, Lei Mingqin, Zhang Shuangyi, Yidan Yu, Zhou Teng, Qin Jing (2020) A noise-immune lstm network for short-term traffic flow forecasting. Chaos 30(3):1–10
35.
Zurück zum Zitat Zhou Teng, Jiang Dazhi, Lin Zhizhe, Han Guoqiang, Xuemiao Xu, Qin Jing (2019) Hybrid dual kalman filtering model for short-term traffic flow forecasting. IET Intell Transp Syst 13(6):1023–1032 Zhou Teng, Jiang Dazhi, Lin Zhizhe, Han Guoqiang, Xuemiao Xu, Qin Jing (2019) Hybrid dual kalman filtering model for short-term traffic flow forecasting. IET Intell Transp Syst 13(6):1023–1032
36.
Zurück zum Zitat Bai Weihua, Zhu Jiaxian, Zhang Huibing, Lin Weiwei, Xi Jianqing (2019) A multi-dimensional resource scheduling strategy based on multilateral complementarity. IEEE Access 7:88481–88503 Bai Weihua, Zhu Jiaxian, Zhang Huibing, Lin Weiwei, Xi Jianqing (2019) A multi-dimensional resource scheduling strategy based on multilateral complementarity. IEEE Access 7:88481–88503
37.
Zurück zum Zitat Lin Miao, Xi Jianqing, Bai Weihua, Jiayin Wu (2019) Ant colony algorithm for multi-objective optimization of container-based microservice scheduling in cloud. IEEE Access 7:83088–83100 Lin Miao, Xi Jianqing, Bai Weihua, Jiayin Wu (2019) Ant colony algorithm for multi-objective optimization of container-based microservice scheduling in cloud. IEEE Access 7:83088–83100
Metadaten
Titel
A cost saving and load balancing task scheduling model for computational biology in heterogeneous cloud datacenters
verfasst von
Wenwei Cai
Jiaxian Zhu
Weihua Bai
Weiwei Lin
Naqin Zhou
Keqin Li
Publikationsdatum
26.05.2020
Verlag
Springer US
Erschienen in
The Journal of Supercomputing / Ausgabe 8/2020
Print ISSN: 0920-8542
Elektronische ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-020-03305-y

Weitere Artikel der Ausgabe 8/2020

The Journal of Supercomputing 8/2020 Zur Ausgabe