Muhammad H. Hilman
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Workflows are an application model that enables the automated execution of multiple interdependent and interconnected tasks. They are widely used by the scientific community to manage the distributed execution and dataflow of complex simulations and experiments. As the popularity of scientific workflows continue to rise, and their computational requirements continue to increase, the emergence and adoption of multi-tenant computing platforms that offer the execution of these workflows as a service becomes widespread. This article discusses the scheduling and resource provisioning problems particular to this type of platform. It presents a detailed taxonomy and a comprehensive survey of the current literature and identifies future directions to foster research in the field of multiple workflow scheduling in multi-tenant distributed computing systems.
Supplemental Material
Available for Download
zip
hilman.zip (115.8 KB)
Supplemental movie, appendix, image and software files for, MultipleWorkflows Scheduling in Multi-tenant Distributed Systems: A Taxonomy and Future Directions
- Roger Barga and Dennis Gannon. 2007. Scientific versus Business Workflows. Springer, London, 9--16. DOI:http://dx.doi.org/10.1007/978-1-84628-757-2_2Google Scholar
- Ian J. Taylor, Ewa Deelman, Dennis B. Gannon, and Matthew Shields. 2014. Workflows for e-Science: Scientific Workflows for Grids. Springer.Google ScholarDigital Library
- Carlos Goncalves, Luis Assuncao, and Jose C. Cunha. 2012. Data analytics in the cloud with flexible MapReduce workflows. In Proceedings of the 4th IEEE International Conference on Cloud Computing Technology and Science. 427--434. DOI:http://dx.doi.org/10.1109/CloudCom.2012.6427527Google ScholarDigital Library
- Jia Yu and Rajkumar Buyya. 2005. A taxonomy of workflow management systems for grid computing. J. Grid Comput. 3, 3 (2005), 171--200. DOI:http://dx.doi.org/10.1007/s10723-005-9010-8Google ScholarCross Ref
- Marek Wieczorek, Andreas Hoheisel, and Radu Prodan. 2008. Taxonomies of the Multi-criteria Grid Workflow Scheduling Problem. Springer US, 237--264. DOI:http://dx.doi.org/10.1007/978-0-387-78446-5_16Google Scholar
- Fuhui Wu, Qingbo Wu, and Yusong Tan. 2015. Workflow scheduling in cloud: A survey. J. Supercomput. 71, 9 (2015), 3373--3418. DOI:http://dx.doi.org/10.1007/s11227-015-1438-4Google ScholarDigital Library
- Sukhpal Singh and Inderveer Chana. 2016. A survey on resource scheduling in cloud computing: Issues and challenges. J. Grid Comput. 14, 2 (2016), 217--264. DOI:http://dx.doi.org/10.1007/s10723-015-9359-2Google ScholarDigital Library
- Ehab N. Alkhanak, Sai P. Lee, and Saif U. R. Khan. 2015. Cost-aware challenges for workflow scheduling approaches in cloud computing environments: Taxonomy and opportunities. Fut. Gener. Comput. Syst. 50, Suppl. C (2015), 3--21. DOI:http://dx.doi.org/10.1016/j.future.2015.01.007Google ScholarDigital Library
- Sucha Smanchat and Kanchana Viriyapant. 2015. Taxonomies of workflow scheduling problem and techniques in the cloud. Fut. Gener. Comput. Syst. 52, Suppl. C (2015), 1--12. DOI:http://dx.doi.org/10.1016/j.future.2015.04.019Google Scholar
- Maria A. Rodriguez and Rajkumar Buyya. 2017. A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments. Concurr. Comput.: Pract. Exp. 29, 8 (2017), e4041--n/a. DOI:http://dx.doi.org/10.1002/cpe.4041Google ScholarCross Ref
- Philipp Leitner and Jürgen Cito. 2016. Patterns in the Chaos: A study of performance variation and predictability in public IaaS clouds. ACM Trans. Internet Technol. 16, 3 (2016), 1--23. DOI:http://dx.doi.org/10.1145/2885497Google ScholarDigital Library
- Saima G. Ahmad, Chee S. Liew, Muhammad M. Rafique, Ehsan U. Munir, and Samee U. Khan. 2014. Data-intensive workflow optimization based on application task graph partitioning in heterogeneous computing systems. In Proceedings of the 4th IEEE International Conference on Big Data and Cloud Computing. 129--136. DOI:http://dx.doi.org/10.1109/BDCloud.2014.63Google Scholar
- Kathy Svitil. 2016. Gravitational waves detected 100 years after Einstein’s prediction. (2016). Retrieved from http://www.caltech.edu/news/gravitational-waves-detected-100-years-after-einstein-s-prediction-49777.Google Scholar
- Jun Qin and Thomas Fahringer. 2012. Scientific Workflows: Programming, Optimization, and Synthesis with ASKALON and AWDL. Springer Science 8 Business Media.Google ScholarCross Ref
- Maria A. Rodriguez and Rajkumar Buyya. 2017. Scientific workflow management system for clouds. In Software Architecture for Big Data and the Cloud, Ivan Mistrik, Rami Bahsoon, Nour Ali, Maritta Heisel, and Bruce Maxim (Eds.). Morgan Kaufmann, Boston, 367--387. DOI:http://dx.doi.org/10.1016/B978-0-12-805467-3.00018-1Google Scholar
- Bartosz Balis. 2016. HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflows. Fut. Gener. Comput. Syst. 55 (2016), 147--162. DOI:http://dx.doi.org/10.1016/j.future.2015.08.015Google ScholarDigital Library
- Prakashan Korambath, Jianwu Wang, Ankur Kumar, Lorin Hochstein, Brian Schott, Robert Graybill, Michael Baldea, and Jim Davis. 2014. Deploying kepler workflows as services on a cloud infrastructure for smart manufacturing. Proc. Comput. Sci. 29 (2014), 2254--2259. DOI:http://dx.doi.org/10.1016/j.procs.2014.05.210Google ScholarCross Ref
- E. Deelman, K. Vahi, M. Rynge, R. Mayani, R. F. da Silva, G. Papadimitriou, and M. Livny. 2019. The evolution of the pegasus workflow management software. Comput. Sci. Eng. 21, 4 (2019), 22--36.Google ScholarCross Ref
- Katherine Wolstencroft, Robert Haines, Donal Fellows, Alan Williams, David Withers, Stuart Owen, Stian Soiland-Reyes, Ian Dunlop, Aleksandra Nenadic, Paul Fisher, Jiten Bhagat, Khalid Belhajjame, Finn Bacall, Alex Hardisty, Abraham Nieva de la Hidalga, Maria P. Balcazar Vargas, Shoaib Sufi, and Carole Goble. 2013. The Taverna workflow suite: Designing and executing workflows of web services on the desktop, web or in the cloud. Nucleic Acids Res. 41, 1 (2013), 557--561. DOI:http://dx.doi.org/10.1093/nar/gkt328Google ScholarCross Ref
- Bill Howe, Garret Cole, Emad Souroush, Paraschos Koutris, Alicia Key, Nodira Khoussainova, and Leilani Battle. 2011. Database-as-a-service for long-tail science. In Scientific and Statistical Database Management. Springer, Berlin, 480--489.Google Scholar
- Jianwu Wang, Prakashan Korambath, Ilkay Altintas, Jim Davis, and Daniel Crawl. 2014. Workflow as a service in the cloud: Architecture and scheduling algorithms. Proced. Comput. Sci. 29, Suppl. C (2014), 546--556. DOI:http://dx.doi.org/10.1016/j.procs.2014.05.049Google ScholarCross Ref
- Sérgio Esteves and Luís Veiga. 2016. WaaS: Workflow-as-a-service for the cloud with scheduling of continuous and data-intensive workflows. Comput. J. 59, 3 (2016), 371--383. DOI:http://dx.doi.org/10.1093/comjnl/bxu158Google ScholarCross Ref
- Bhaskar P. Rimal and Martin Maier. 2017. Workflow scheduling in multi-tenant cloud computing environments. IEEE Trans. Parallel Distrib. Syst. 28, 1 (2017), 290--304. DOI:http://dx.doi.org/10.1109/TPDS.2016.2556668Google ScholarDigital Library
- Gideon Juve, Ann Chervenak, Ewa Deelman, Shishir Bharathi, Gaurang Mehta, and Karan Vahi. 2013. Characterizing and profiling scientific workflows. Fut. Gener. Comput. Syst. 29, 3 (2013), 682--692. DOI:http://dx.doi.org/10.1016/j.future.2012.08.015Google ScholarDigital Library
- Ewa Deelman, Gurmeet Singh, Miron Livny, Bruce Berriman, and John Good. 2008. The cost of doing science on the cloud: The montage example. In Proceedings of the ACM/IEEE Conference on Supercomputing. 1--12. DOI:http://dx.doi.org/10.1109/SC.2008.5217932Google ScholarCross Ref
- Philip Maechling, Ewa Deelman, Li Zhao, Robert Graves, Gaurang Mehta, Nitin Gupta, John Mehringer, Carl Kesselman, Scott Callaghan, David Okaya, Hunter Francoeur, Vipin Gupta, Yifeng Cui, Karan Vahi, Thomas Jordan, and Edward Field. 2007. SCEC CyberShake Workflows—Automating Probabilistic Seismic Hazard Analysis Calculations. Springer, London, 143--163. DOI:http://dx.doi.org/10.1007/978-1-84628-757-2_10Google Scholar
- Phuong Nguyen and Klara Nahrstedt. 2017. MONAD: Self-adaptive micro-service infrastructure for heterogeneous scientific workflows. In Proceedings of the 2017 IEEE International Conference on Autonomic Computing. 187--196. DOI:http://dx.doi.org/10.1109/ICAC.2017.38Google ScholarCross Ref
- Lincoln Bryant, Jeremy Van, Benedikt Riedel, Robert W. Gardner, Jose C. Bejar, John Hover, Ben Tovar, Kenyi Hurtado, and Douglas Thain. 2018. VC3: A virtual cluster service for community computation. In Proceedings of the Practice and Experience on Advanced Research Computing. 30:1--30:8. DOI:http://dx.doi.org/10.1145/3219104.3219125Google Scholar
- Maxim Belkin, Roland Haas, Galen W. Arnold, Hon W. Leong, Eliu A. Huerta, David Lesny, and Mark Neubauer. 2018. Container solutions for HPC systems: A case study of using shifters on blue waters. In Proceedings of Practice and Experience in Advanced Research Computing. 1--8. DOI:https://doi.org/10.1145/3219104.3219145Google ScholarDigital Library
- Carl Witt, Marc Bux, Wladislaw Gusew, and Ulf Leser. 2019. Predictive performance modeling for distributed batch processing using black box monitoring and machine learning. Inf. Syst. 82 (2019), 33--52. DOI:http://dx.doi.org/10.1016/j.is.2019.01.006Google ScholarDigital Library
- Farrukh Nadeem and Thomas Fahringer. 2009. using templates to predict execution time of scientific workflow applications in the grid. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid. 316--323. DOI:http://dx.doi.org/10.1109/CCGRID.2009.77Google ScholarDigital Library
- Rafael F. da Silva, Gideon Juve, Mats Rynge, Ewa Deelman, and Miron Livny. 2015. Online task resource consumption prediction for scientific workflows. Parallel Process. Lett. 25, 3 (2015), 1541003. DOI:http://dx.doi.org/10.1142/S0129626415410030Google ScholarCross Ref
- Thanh P. Pham, Juan J. Durillo, and Thomas Fahringer. 2017. Predicting workflow task execution time in the cloud using a two-stage machine learning approach. IEEE Trans. Cloud Comput. 99 (2017), 1--1. DOI:http://dx.doi.org/10.1109/TCC.2017.2732344Google ScholarCross Ref
- Keith R. Jackson, Lavanya Ramakrishnan, Krishna Muriki, Shane Canon, Shreyas Cholia, John Shalf, Harvey J. Wasserman, and Nicholas J. Wright. 2010. Performance analysis of high performance computing applications on the amazon web services cloud. In Proceedings of the 2nd IEEE International Conference on Cloud Computing Technology and Science. 159--168. DOI:http://dx.doi.org/10.1109/CloudCom.2010.69Google Scholar
- Ming Mao and Marty Humphrey. 2012. A performance study on the VM startup time in the cloud. In Proceedings of the 5th IEEE International Conference on Cloud Computing. 423--430. DOI:http://dx.doi.org/10.1109/CLOUD.2012.103Google ScholarDigital Library
- Mike Jones, Bill Arcand, Bill Bergeron, David Bestor, Chansup Byun, Lauren Milechin, Vijay Gadepally, Matt Hubbell, Jeremy Kepner, Pete Michaleas, Julie Mullen, Andy Prout, Tony Rosa, Siddharth Samsi, Charles Yee, and Albert Reuther. 2016. Scalability of VM provisioning systems. In Proceedings of the IEEE High Performance Extreme Computing Conference. 1--5. DOI:http://dx.doi.org/10.1109/HPEC.2016.7761629Google ScholarCross Ref
- Michael A. Murphy, Brandon Kagey, Michael Fenn, and Sebastien Goasguen. 2009. Dynamic provisioning of virtual organization clusters. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid. 364--371. DOI:http://dx.doi.org/10.1109/CCGRID.2009.37Google ScholarDigital Library
- Wei Chen, Young C. Lee, Alan Fekete, and Albert Y. Zomaya. 2015. Adaptive multiple-workflow scheduling with task rearrangement. J. Supercomput. 71, 4 (2015), 1297--1317. DOI:http://dx.doi.org/10.1007/s11227-014-1361-0Google ScholarDigital Library
- Yirong Wang, Kuochan Huang, and Fengjian Wang. 2016. Scheduling online mixed-parallel workflows of rigid tasks in heterogeneous multi-cluster environments. Fut. Gener. Comput. Syst. 60, Suppl. C (2016), 35--47. DOI:http://dx.doi.org/10.1016/j.future.2016.01.013Google ScholarDigital Library
- Hamid Arabnejad, Jorge G. Barbosa, and Frédéric Suter. 2014. Fair resource sharing for dynamic scheduling of workflows on heterogeneous systems. In High-Performance Computing on Complex Environments.Google Scholar
- Amelia C. Zhou, Bingsheng He, and Cheng Liu. 2016. Monetary cost optimizations for hosting workflow-as-a-service in IaaS clouds. IEEE Trans. Cloud Comput. 4, 1 (2016), 34--48. DOI:http://dx.doi.org/10.1109/TCC.2015.2404807Google ScholarDigital Library
- Zhifeng Yu and Weisong Shi. 2008. A planner-guided scheduling strategy for multiple workflow applications. In Proceedings of the International Conference on Parallel Processing. 1--8. DOI:http://dx.doi.org/10.1109/ICPP-W.2008.10Google ScholarDigital Library
- Haluk Topcuoglu, Salim Hariri, and Min You Wu. 2002. Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13, 3 (2002), 260--274. DOI:http://dx.doi.org/10.1109/71.993206Google ScholarDigital Library
- Meng Xu, Lizhen Cui, Haiyang Wang, and Yanbing Bi. 2009. A multiple QoS constrained scheduling strategy of multiple workflows for cloud computing. In Proceedings of the IEEE International Symposium on Parallel and Distributed Processing with Applications. 629--634. DOI:http://dx.doi.org/10.1109/ISPA.2009.95Google ScholarCross Ref
- Cui Lizhen, Xu Meng, and Yanbing Bi. 2009. A scheduling strategy for multiple QoS constrained grid workflows. In Proceedings of the Joint Conferences on Pervasive Computing. 561--566. DOI:http://dx.doi.org/10.1109/JCPC.2009.5420119Google ScholarCross Ref
- Jorge G. Barbosa and Belmiro Moreira. 2011. Dynamic scheduling of a batch of parallel task jobs on heterogeneous clusters. Parallel Comput. 37, 8 (2011), 428--438. DOI:http://dx.doi.org/10.1016/j.parco.2010.12.004Google ScholarDigital Library
- Jorge G. Barbosa, Celeste Morais, Ruben Nobrega, and Antònio Monteiro. 2005. Static scheduling of dependent parallel tasks on heterogeneous clusters. In Proceedings of the IEEE International Conference on Cluster Computing. 1--8. DOI:http://dx.doi.org/10.1109/CLUSTR.2005.347024Google ScholarCross Ref
- Chihchiang Hsu, Kuochan Huang, and Fengjian Wang. 2011. Online scheduling of workflow applications in grid environments. Fut. Gener. Comput. Syst. 27, 6 (2011), 860--870. DOI:http://dx.doi.org/10.1016/j.future.2010.10.015Google ScholarDigital Library
- Hamid Arabnejad and Jorge G. Barbosa. 2017. Maximizing the completion rate of concurrent scientific applications under time and budget constraints. J. Comput. Sci. 23, Suppl. C (2017), 120--129. DOI:http://dx.doi.org/10.1016/j.jocs.2016.10.013Google ScholarCross Ref
- Hamid Arabnejad and Jorge G. Barbosa. 2017. Multi-QoS constrained and profit-aware scheduling approach for concurrent workflows on heterogeneous systems. Fut. Gener. Comput. Syst. 68, Suppl. C (2017), 211--221. DOI:http://dx.doi.org/10.1016/j.future.2016.10.003Google ScholarCross Ref
- Georgios L. Stavrinides and Helen D. Karatza. 2011. Scheduling multiple task graphs in heterogeneous distributed real-time systems by exploiting schedule holes with bin packing techniques. Simul. Model. Pract. Theory 19, 1 (2011), 540--552. DOI:http://dx.doi.org/10.1016/j.simpat.2010.08.010Google ScholarCross Ref
- Georgios L. Stavrinides and Helen D. Karatza. 2015. A cost-effective and QoS-aware approach to scheduling real-time workflow applications in PaaS and SaaS clouds. In Proceedings of the 3rd International Conference on Future Internet of Things and Cloud. 231--239. DOI:http://dx.doi.org/10.1109/FiCloud.2015.93Google Scholar
- Yuxin Wang, Shijie Cao, Guan Wang, Zhen Feng, Chi Zhang, and He Guo. 2017. Fairness scheduling with dynamic priority for multi workflow on heterogeneous systems. In Proceedings of the 2nd IEEE International Conference on Cloud Computing and Big Data Analysis. 404--409. DOI:http://dx.doi.org/10.1109/ICCCBDA.2017.7951947Google ScholarCross Ref
- Guoqi Xie, Liangjiao Liu, Liu Yang, and Renfa Li. 2017. Scheduling trade-off of dynamic multiple parallel workflows on heterogeneous distributed computing systems. Concurr. Comput.: Pract. Exp. 29, 2 (2017), e3782--n/a. DOI:http://dx.doi.org/10.1002/cpe.3782Google ScholarCross Ref
- Yinglin Tsai, Hsiaoching Liu, and Kuochan Huang. 2015. Adaptive dual-criteria task group allocation for clustering-based multi-workflow scheduling on parallel computing platform. J. Supercomput. 71, 10 (2015), 3811--3831. DOI:http://dx.doi.org/10.1007/s11227-015-1469-xGoogle ScholarDigital Library
- Bing Lin, Wenzhong Guo, and Xiuyan Lin. 2016. Online optimization scheduling for scientific workflows with deadline constraint on hybrid clouds. Concurr. Comput.: Pract. Exp. 28, 11 (2016), 3079--3095. DOI:http://dx.doi.org/10.1002/cpe.3582Google ScholarDigital Library
- Shaghayegh Sharif, Javid Taheri, Albert Y. Zomaya, and Surya Nepal. 2014. Online multiple workflow scheduling under privacy and deadline in hybrid cloud environment. In Proceedings of the 6th IEEE International Conference on Cloud Computing Technology and Science. 455--462. DOI:http://dx.doi.org/10.1109/CloudCom.2014.128Google ScholarDigital Library
- Maria A. Rodriguez and Rajkumar Buyya. 2018. Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Fut. Gener. Comput. Syst. 79, 2 (2018), 739--750. DOI:http://dx.doi.org/10.1016/j.future.2017.05.009Google ScholarDigital Library
- Xiaomin Zhu, Ji Wang, Hui Guo, Dakai Zhu, Laurence T. Yang, and Ling Liu. 2016. Fault-tolerant scheduling for real-time scientific workflows with elastic resource provisioning in virtualized clouds. IEEE Trans. Parallel Distrib. Syst. 27, 12 (2016), 3501--3517. DOI:http://dx.doi.org/10.1109/TPDS.2016.2543731Google ScholarDigital Library
- Xiaolong Xu, Wanchun Dou, Xuyun Zhang, and Jinjun Chen. 2016. EnReal: An energy-aware resource allocation method for scientific workflow executions in cloud environment. IEEE Trans. Cloud Comput. 4, 2 (2016), 166--179. DOI:http://dx.doi.org/10.1109/TCC.2015.2453966Google ScholarCross Ref
- Huangke Chen, Xiaomin Zhu, Dishan Qiu, Hui Guo, Laurence T. Yang, and Peizhong Lu. 2016. EONS: Minimizing energy consumption for executing real-time workflows in virtualized cloud data centers. In Proceedings of the 45th International Conference on Parallel Processing Workshops. 385--392. DOI:http://dx.doi.org/10.1109/ICPPW.2016.60Google ScholarCross Ref
- Guoqi Xie, Gang Zeng, Junqiang Jiang, Chunnian Fan, Renfa Li, and Keqin Li. 2017. Energy management for multiple real-time workflows on cyber--physical cloud systems. Fut. Gener. Comput. Syst. (2017). DOI:http://dx.doi.org/10.1016/j.future.2017.05.033Google Scholar
- Huangke Chen, Xiaomin Zhu, Dishan Qiu, and Ling Liu. 2016. Uncertainty-aware real-time workflow scheduling in the cloud. In Proceedings of the 9th IEEE International Conference on Cloud Computing. 577--584. DOI:http://dx.doi.org/10.1109/CLOUD.2016.0082Google ScholarCross Ref
- Huangke Chen, Jianghan Zhu, Zhenshi Zhang, Manhao Ma, and Xin Shen. 2017. Real-time workflows oriented online scheduling in uncertain cloud environment. J. Supercomput. 73, 11 (2017), 4906--4922. DOI:http://dx.doi.org/10.1007/s11227-017-2060-4Google ScholarDigital Library
- Huangke Chen, Xiaomin Zhu, Guipeng Liu, and Witold Pedrycz. 2018. Uncertainty-aware online scheduling for real-time workflows in cloud service environment. IEEE Trans. Serv. Comput. (2018), 1--1. DOI:http://dx.doi.org/10.1109/TSC.2018.2866421Google Scholar
- Jiagang Liu, Ju Ren, Wei Dai, Deyu Zhang, Pude Zhou, Yaoxue Zhang, Geyong Min, and Noushin Najjari. 2019. Online multi-workflow scheduling under uncertain task execution time in IaaS clouds. IEEE Trans. Cloud Comput. (2019), 1--1. DOI:http://dx.doi.org/10.1109/TCC.2019.2906300Google ScholarCross Ref
- Georgios L. Stavrinides, Francisco R. Duro, Helen D. Karatza, Javier G. Blas, and Jesus Carretero. 2017. Different aspects of workflow scheduling in large-scale distributed systems. Simul. Model. Pract. Theory 70, Suppl. C (2017), 120--134. DOI:http://dx.doi.org/10.1016/j.simpat.2016.10.009Google ScholarCross Ref
- Naqin Zhou, FuFang Li, Kefu Xu, and Deyu Qi. 2018. Concurrent workflow budget- and deadline-constrained scheduling in heterogeneous distributed environments. Soft Computing 22, 23 (2018), 7705--7718. DOI:http://dx.doi.org/10.1007/s00500-018-3229-3Google ScholarDigital Library
- Huangke Chen, Jianghan Zhu, Guohua Wu, and Lisu Huo. 2018. Cost-efficient reactive scheduling for real-time workflows in clouds. J. Supercomput. 74, 11 (2018), 6291--6309. DOI:http://dx.doi.org/10.1007/s11227-018-2561-9Google ScholarDigital Library
- Georgios L. Stavrinides and Helen D. Karatza. 2010. Scheduling multiple task graphs with end-to-end deadlines in distributed real-time systems utilizing imprecise computations. J. Syst. Softw. 83, 6 (2010), 1004--1014. DOI:http://dx.doi.org/10.1016/j.jss.2009.12.025Google ScholarDigital Library
- Francisco R. Duro, Javier G. Blas, and Jesus Carretero. 2013. A hierarchical parallel storage system based on distributed memory for large scale systems. In Proceedings of the 20th European MPI Users’ Group Meeting. 139--140. DOI:http://dx.doi.org/10.1145/2488551.2488598Google ScholarDigital Library
- Xiaoyong Tang, Kenli Li, Guiping Liao, Kui Fang, and Fan Wu. 2011. A stochastic scheduling algorithm for precedence constrained tasks on grid. Fut. Gener. Comput. Syst. 27, 8 (2011), 1083--1091. DOI:http://dx.doi.org/10.1016/j.future.2011.04.007Google ScholarDigital Library
- Deepak Poola, Saurab Garg, Rajkumar Buyya, Yun Yang, and Kotagiri Ramamohanarao. 2014. Robust scheduling of scientific workflows with deadline and budget constraints in clouds. In Proceedings of the 28th IEEE International Conference on Advanced Information Networking and Applications. 858--865. DOI:http://dx.doi.org/10.1109/AINA.2014.105Google ScholarDigital Library
- Vahid Ebrahimirad, Maziar Goudarzi, and Aboozar Rajabi. 2015. Energy-aware scheduling for precedence-constrained parallel virtual machines in virtualized data centers. J. Grid Comput. 13, 2 (2015), 233--253. DOI:http://dx.doi.org/10.1007/s10723-015-9327-xGoogle ScholarDigital Library
- Ilia Pietri and Rizos Sakellariou. 2014. Energy-aware workflow scheduling using frequency scaling. In Proceedings of the 43rd International Conference on Parallel Processing Workshops. 104--113. DOI:http://dx.doi.org/10.1109/ICPPW.2014.26Google ScholarDigital Library
- Xiao Qin and Hong Jiang. 2006. A novel fault-tolerant scheduling algorithm for precedence constrained tasks in real-time heterogeneous systems. Parallel Comput. 32, 5 (2006), 331--356. DOI:http://dx.doi.org/10.1016/j.parco.2006.06.006Google ScholarDigital Library
- Shuo Zhang, Baosheng Wang, Baokang Zhao, and Jing Tao. 2013. An energy-aware task scheduling algorithm for a heterogeneous data center. In Proceedings of the 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications. 1471--1477. DOI:http://dx.doi.org/10.1109/TrustCom.2013.178Google ScholarDigital Library
- Juan J. Durillo, Hamid M. Fard, and Radu Prodan. 2012. MOHEFT: A multi-objective list-based method for workflow scheduling. In Proceedings of the 4th IEEE International Conference on Cloud Computing Technology and Science. 185--192. DOI:http://dx.doi.org/10.1109/CloudCom.2012.6427573Google ScholarDigital Library
- Guo Zhong Tian, Chuang Bai Xiao, Zhu Sheng Xu, and Xia Xiao. 2012. Hybrid scheduling strategy for multiple DAGs workflow in heterogeneous system. J. Softw. 23, 10 (2012), 2720--2734.Google ScholarCross Ref
- Guoqi Xie, Renfa Li, Xiongren Xiao, and Yuekun Chen. 2014. A high-performance DAG task scheduling algorithm for heterogeneous networked embedded systems. In Proceedings of the 28th IEEE International Conference on Advanced Information Networking and Applications. 1011--1016. DOI:http://dx.doi.org/10.1109/AINA.2014.123Google ScholarDigital Library
- Zhuo Tang, Ling Qi, Zhenzhen Cheng, Kenli Li, Samee U. Khan, and Keqin Li. 2016. An energy-efficient task scheduling algorithm in DVFS-enabled cloud environment. J. Grid Comput. 14, 1 (2016), 55--74. DOI:http://dx.doi.org/10.1007/s10723-015-9334-yGoogle ScholarDigital Library
- Ewa Deelman, Tom Peterka, Ilkay Altintas, Christopher D. Carothers, Kerstin K. van Dam, Kenneth Moreland, Manish Parashar, Lavanya Ramakrishnan, Michela Taufer, and Jeffrey Vetter. 2018. The future of scientific workflows. Int. J. High Perf. Comput. Appl. 32, 1 (2018), 159--175. DOI:http://dx.doi.org/10.1177/1094342017704893Google ScholarDigital Library
- Maria Fazio, Antonio Celesti, Rajiv Ranjan, Chang Liu, Lydia Chen, and Massimo Villari. 2016. Open issues in scheduling microservices in the cloud. IEEE Cloud Comput. 3, 5 (2016), 81--88. DOI:http://dx.doi.org/10.1109/MCC.2016.112Google ScholarCross Ref
- Zhanibek Kozhirbayev and Richard O. Sinnott. 2017. A performance comparison of container-based technologies for the cloud. Fut. Gener. Comput. Syst. 68, Suppl. C (2017), 175--182. DOI:http://dx.doi.org/10.1016/j.future.2016.08.025Google ScholarCross Ref
- Wolfgang Gerlach, Wei Tang, Andreas Wilke, Dan Olson, and Folker Meyer. 2015. Container orchestration for scientific workflows. In Proceedings of the IEEE International Conference on Cloud Engineering. 377--378. DOI:http://dx.doi.org/10.1109/IC2E.2015.87Google ScholarDigital Library
- Rawaa Qasha, Jacek Cala, and Paul Watson. 2016. Dynamic deployment of scientific workflows in the cloud using container virtualization. In Proceedings of the IEEE International Conference on Cloud Computing Technology and Science. 269--276. DOI:http://dx.doi.org/10.1109/CloudCom.2016.0052Google ScholarCross Ref
- Kai Liu, Kento Aida, Shigetoshi Yokoyama, and Yoshinobu Masatani. 2016. Flexible container-based computing platform on cloud for scientific workflows. In Proceedings of the International Conference on Cloud Computing Research and Innovations. 56--63. DOI:http://dx.doi.org/10.1109/ICCCRI.2016.17Google ScholarCross Ref
- Eidah J. Alzahrani, Zahir Tari, Young C. Lee, Deafallah Alsadie, and Albert Y. Zomaya. 2017. adCFS: Adaptive completely fair scheduling policy for containerised workflows systems. In Proceedings of the 16th IEEE International Symposium on Network Computing and Applications. 1--8. DOI:http://dx.doi.org/10.1109/NCA.2017.8171362Google Scholar
- Theo Combe, Antony Martin, and Roberto D. Pietro. 2016. To Docker or not to Docker: A security perspective. IEEE Cloud Comput. 3, 5 (2016), 54--62. DOI:http://dx.doi.org/10.1109/MCC.2016.100Google ScholarCross Ref
- Gregory M. Kurtzer, Vanessa Sochat, and Michael W. Bauer. 2017. Singularity: Scientific containers for mobility of compute. PLoS ONE 12, 5 (2017), 1--20. DOI:http://dx.doi.org/10.1371/journal.pone.0177459Google ScholarCross Ref
- Emily Le and David Paz. 2017. Performance analysis of applications using singularity container on SDSC comet. In Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact. 66:1--66:4. DOI:http://dx.doi.org/10.1145/3093338.3106737Google Scholar
- Heru Suhartanto, Agung P. Pasaribu, Muhammad F. Siddiq, Muhammad I. Fadhila, Muhammad H. Hilman, and Arry Yanuar. 2017. A preliminary study on shifting from virtual machine to Docker container for insilico drug discovery in the cloud. Int. J. Technol. 8, 4 (2017).Google ScholarCross Ref
- Sareh Fotuhi Piraghaj, Amir Vahid Dastjerdi, Rodrigo N. Calheiros, and Rajkumar Buyya. 2017. ContainerCloudSim: An environment for modeling and simulation of containers in cloud data centers. Softw.: Pract. Exp. 47, 4 (2017), 505--521. DOI:http://dx.doi.org/10.1002/spe.2422Google ScholarCross Ref
- Maciej Malawski. 2016. Towards serverless execution of scientific workflows - HyperFlow case study. In Proceedings of the Workshop of Workflows in Support of Large-Scale Sciences. 25--33.Google Scholar
- Qingye Jiang, Young C. Lee, and Albert Y. Zomaya. 2017. Serverless execution of scientific workflows. In Proceedings of the 15th International Conference Service-Oriented Computing. 706--721. DOI:http://dx.doi.org/10.1007/978-3-319-69035-3_51Google Scholar
- Maciej Malawski, Adam Gajek, Adam Zima, Bartosz Balis, and Kamil Figiela. 2017. Serverless execution of scientific workflows: Experiments with HyperFlow, AWS lambda and Google cloud functions. Fut. Gener. Comput. Syst. (2017). DOI:http://dx.doi.org/10.1016/j.future.2017.10.029Google Scholar
- Josef Spillner, Cristian Mateos, and David A. Monge. 2018. FaaSter, better, cheaper: The prospect of serverless scientific computing and HPC. In High Performance Computing, Esteban Mocskos and Sergio Nesmachnow (Eds.). Springer International Publishing, Cham, 154--168.Google Scholar
- Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: Library operating systems for the cloud. In Proceedings of the 18th International Conference on Architectural Support for Programming Languages and Operating Systems. 461--472. DOI:http://dx.doi.org/10.1145/2451116.2451167Google ScholarDigital Library
- Dan Williams, Ricardo Koller, Martin Lucina, and Nikhil Prakash. 2018. Unikernels as processes. In Proceedings of the ACM Symposium on Cloud Computing. 199--211. DOI:http://dx.doi.org/10.1145/3267809.3267845Google ScholarDigital Library
- Foued Jrad, Jie Tao, and Achim Streit. 2013. A broker-based framework for multi-cloud workflows. In Proceedings of the International Workshop on Multi-cloud Applications and Federated Clouds. 61--68. DOI:http://dx.doi.org/10.1145/2462326.2462339Google ScholarDigital Library
- Javier D. Montes, Mengsong Zou, Rahul Singh, Shu Tao, and Manish Parashar. 2014. Data-driven workflows in multi-cloud marketplaces. In Proceedings of the 7th IEEE International Conference on Cloud Computing. 168--175. DOI:http://dx.doi.org/10.1109/CLOUD.2014.32Google ScholarDigital Library
- Yushi Omote, Takahiro Shinagawa, and Kazuhiko Kato. 2015. Improving agility and elasticity in bare-metal clouds. In Proceedings of the 20th International Conference on Architectural Support for Programming Languages and Operating Systems. 145--159. DOI:http://dx.doi.org/10.1145/2694344.2694349Google ScholarDigital Library
- Ryan Shea, Feng Wang, Haiyang Wang, and Jiangchuan Liu. 2014. A deep investigation into network performance in virtual machine based cloud environments. In Proceeding of the IEEE Conference on Computer Communications. 1285--1293. DOI:http://dx.doi.org/10.1109/INFOCOM.2014.6848061Google ScholarCross Ref
- Farrukh Nadeem, Daniyal Alghazzawi, Abdulfattah Mashat, Khalid Fakeeh, Abdullah Almalaise, and Hani Hagras. 2017. Modeling and predicting execution time of scientific workflows in the grid using radial basis function neural network. Cluster Comput. 20, 3 (2017), 2805--2819. DOI:http://dx.doi.org/10.1007/s10586-017-1018-xGoogle ScholarDigital Library
- Doyen Sahoo, Steven C. H. Hoi, and Bin Li. 2019. Large scale online multiple kernel regression with application to time-series prediction. ACM Trans. Knowl. Discov. Data 13, 1 (2019), 9:1--9:33. DOI:http://dx.doi.org/10.1145/3299875Google ScholarDigital Library
- Muhammad H. Hilman, Maria A. Rodríguez, and Rajkumar Buyya. 2018. Task runtime prediction in scientific workflows using an online incremental learning approach. In Proceedings of the 11th IEEE/ACM International Conference on Utility and Cloud Computing. 93--102. DOI:http://dx.doi.org/10.1109/UCC.2018.00018Google ScholarCross Ref
- Jan Zenisek, Florian Holzinger, and Michael Affenzeller. 2019. Machine learning based concept drift detection for predictive maintenance. Comput. Industr. Eng. 137 (2019), 106031. DOI:http://dx.doi.org/10.1016/j.cie.2019.106031Google ScholarCross Ref
- Taghrid Samak, Dan Gunter, Monte Goode, Ewa Deelman, Gideon Juve, Gaurang Mehta, Fabio Silva, and Karan Vahi. 2011. Online fault and anomaly detection for large-scale scientific workflows. In Proceedings of the IEEE International Conference on High Performance Computing and Communications. 373--381. DOI:http://dx.doi.org/10.1109/HPCC.2011.55Google ScholarDigital Library
- Prathamesh Gaikwad, Anirban Mandal, Paul Ruth, Gideon Juve, Dariusz Król, and Ewa Deelman. 2016. Anomaly detection for scientific workflow applications on networked clouds. In Proceedings of the International Conference on High Performance Computing Simulation. 645--652. DOI:http://dx.doi.org/10.1109/HPCSim.2016.7568396Google ScholarCross Ref
- Maria A. Rodriguez, Ramamohanarao Kotagiri, and Rajkumar Buyya. 2018. Detecting performance anomalies in scientific workflows using hierarchical temporal memory. Fut. Gener. Comput. Syst. 88 (2018), 624--635. DOI:http://dx.doi.org/10.1016/j.future.2018.05.014Google ScholarCross Ref
- Hamid Arabnejad and Jorge G. Barbosa. 2015. Multi-workflow QoS-constrained scheduling for utility computing. In Proceedings of the 18th IEEE International Conference on Computational Science and Engineering. 137--144. DOI:http://dx.doi.org/10.1109/CSE.2015.29Google Scholar
- Mozhgan Ghasemzadeh, Hamid Arabnejad, and Jorge G. Barbosa. 2017. Deadline-budget constrained scheduling algorithm for scientific workflows in a cloud environment. In Proceedings of the 20th International Conference on Principles of Distributed Systems, Vol. 70. 19:1--19:16. DOI:http://dx.doi.org/10.4230/LIPIcs.OPODIS.2016.19Google Scholar
- Hamid M. Fard, Radu Prodan, Juan J. Durillo, and Thomas Fahringer. 2012. A multi-objective approach for workflow scheduling in heterogeneous environments. In Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 300--309. DOI:http://dx.doi.org/10.1109/CCGrid.2012.114Google ScholarDigital Library
- Maciej Malawski, Gideon Juve, Ewa Deelman, and Jarek Nabrzyski. 2015. Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. Fut. Gener. Comput. Syst. 48, Suppl. C (2015), 1--18. DOI:http://dx.doi.org/10.1016/j.future.2015.01.004Google Scholar
- Anton Beloglazov, Rajkumar Buyya, Young C. Lee, and Albert Y. Zomaya. 2011. Chapter 3 - A taxonomy and survey of energy-efficient data centers and cloud computing systems. Advances in Computers, Vol. 82. Elsevier, 47--111. DOI:http://dx.doi.org/h10.1016/B978-0-12-385512-1.00003-7Google Scholar
- Adel Nadjaran Toosi, Chenhao Qu, Marcos Dias de Assunção, and Rajkumar Buyya. 2017. Renewable-aware geographical load balancing of web applications for sustainable data centers. J. Netw. Comput. Appl. 83, C (2017), 155--168. DOI:http://dx.doi.org/10.1016/j.jnca.2017.01.036Google Scholar
- Lingfang Zeng, Bharadwaj Veeravalli, and Xiaorong Li. 2015. SABA: A security-aware and budget-aware workflow scheduling strategy in clouds. J. Parallel Distrib. Comput. 75 (2015), 141--151. DOI:http://dx.doi.org/10.1016/j.jpdc.2014.09.002Google ScholarDigital Library
- Feng Zhao, Chao Li, and Chunfeng Liu. 2014. A cloud computing security solution based on fully homomorphic encryption. In Proceedings of the 16th International Conference on Advanced Communication Technology. 485--488. DOI:http://dx.doi.org/10.1109/ICACT.2014.6779008Google ScholarCross Ref
- Jayavardhana Gubbi, Rajkumar Buyya, Slaven Marusic, and Marimuthu Palaniswami. 2013. Internet of Things (IoT): A vision, architectural elements, and future directions. Fut. Gener. Comput. Syst. 29, 7 (2013), 1645--1660. DOI:http://dx.doi.org/10.1016/j.future.2013.01.010Google ScholarDigital Library
- Charalampos Doukas and Fabio Antonelli. 2014. A full end-to-end platform as a service for smart city applications. In Proceedings of the 10th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications. 181--186. DOI:http://dx.doi.org/10.1109/WiMOB.2014.6962168Google ScholarCross Ref
- Matteo Nardelli, Stefan Nastic, Schahram Dustdar, Massimo Villari, and Rajiv Ranjan. 2017. Osmotic flow: Osmotic computing + IoT workflow. IEEE Cloud Comput. 4, 2 (2017), 68--75. DOI:http://dx.doi.org/10.1109/MCC.2017.22Google ScholarCross Ref
- Georgios L. Stavrinides and Helen D. Karatza. 2019. A hybrid approach to scheduling real-time IoT workflows in fog and cloud environments. Multimedia Tools Appl. 78, 17 (2019), 24639--24655. DOI:http://dx.doi.org/10.1007/s11042-018-7051-9Google ScholarCross Ref
Index Terms
- Multiple Workflows Scheduling in Multi-tenant Distributed Systems: A Taxonomy and Future Directions
Recommendations
Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows
Scientists have many computing infrastructures available to conduct their research, including grids and public or private clouds. This article explores the use of these cyberinfrastructures to execute scientific workflows, an important class of ...
Deadline-constrained algorithms for scheduling of bag-of-tasks and workflows in cloud computing environments
HP3C: Proceedings of the 2nd International Conference on High Performance Compilation, Computing and CommunicationsCloud computing is an emerging distributed computing paradigm that solves immense scientific applications through distributing computing resources over the Internet. These applications may have a huge number of tasks that may increase their execution ...
Bringing Scientific Workflows to Amazon SWF
SEAA '13: Proceedings of the 2013 39th Euromicro Conference on Software Engineering and Advanced ApplicationsIn response to the ever-increasing needs of scientific applications for resources, Cloud computing emerged as an alternative on-demand and cost-effective resource provisioning approach. In this context, Cloud providers have recognised the importance of ...
Comments