Skip to main content

2010 | OriginalPaper | Buchkapitel

13. SwinDeW-C: A Peer-to-Peer Based Cloud Workflow System

verfasst von : Xiao Liu, Dong Yuan, Gaofeng Zhang, Jinjun Chen, Yun Yang

Erschienen in: Handbook of Cloud Computing

Verlag: Springer US

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Workflow systems are designed to support the process automation of large scale business and scientific applications. In recent years, many workflow systems have been deployed on high performance computing infrastructures such as cluster, peer-to-peer (p2p), and grid computing (Moore, 2004; Wang, Jie, & Chen, 2009; Yang, Liu, Chen, Lignier, & Jin, 2007). One of the driving forces is the increasing demand of large scale instance and data/computation intensive workflow applications (large scale workflow applications for short) which are common in both eBusiness and eScience application areas. Typical examples (will be detailed in Section 13.2.1) include such as the transaction intensive nation-wide insurance claim application process; the data and computation intensive pulsar searching process in Astrophysics. Generally speaking, instance intensive applications are those processes which need to be executed for a large number of times sequentially within a very short period or concurrently with a large number of instances (Liu, Chen, Yang, & Jin, 2008; Liu et al., 2010; Yang et al., 2008). Therefore, large scale workflow applications normally require the support of high performance computing infrastructures (e.g. advanced CPU units, large memory space and high speed network), especially when workflow activities are of data and computation intensive themselves. In the real world, to accommodate such a request, expensive computing infrastructures including such as supercomputers and data servers are bought, installed, integrated and maintained with huge cost by system users

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Ardagna, D., & Pernici, B. (2007). Adaptive service composition in flexible processes. IEEE Transactions on Software Engineering, 33(6), 369–384.CrossRef Ardagna, D., & Pernici, B. (2007). Adaptive service composition in flexible processes. IEEE Transactions on Software Engineering, 33(6), 369–384.CrossRef
Zurück zum Zitat Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., et al. (2009). Above the clouds: A Berkeley view of cloud computing (Tech. Rep., University of California, Berkeley). Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., et al. (2009). Above the clouds: A Berkeley view of cloud computing (Tech. Rep., University of California, Berkeley).
Zurück zum Zitat de Assuncao, M. D., di Costanzo, A., & Buyya, R. (2009). Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters. Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing, Garching, Germany, 1–10. de Assuncao, M. D., di Costanzo, A., & Buyya, R. (2009). Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters. Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing, Garching, Germany, 1–10.
Zurück zum Zitat Bhargav-spantzel, A., Squicciarini, A. C., & Bertino, E. (2007). Trust negotiation in identity management. IEEE Security & Privacy, 5(2), 55–63.CrossRef Bhargav-spantzel, A., Squicciarini, A. C., & Bertino, E. (2007). Trust negotiation in identity management. IEEE Security & Privacy, 5(2), 55–63.CrossRef
Zurück zum Zitat Bose, R., & Frew, J. (2005). Lineage retrieval for scientific data processing: A survey. ACM Computing Surveys, 37(1), 1–28.CrossRef Bose, R., & Frew, J. (2005). Lineage retrieval for scientific data processing: A survey. ACM Computing Surveys, 37(1), 1–28.CrossRef
Zurück zum Zitat Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems, 25(6), 599–616.CrossRef Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems, 25(6), 599–616.CrossRef
Zurück zum Zitat Calheiros, R. N., Ranjan, R., De Rose, C. A. F., & Buyya, R. (2009). CloudSim: A novel framework for modeling and simulation of cloud computing infrastructures and services (Tech. Rep., Grid Computing and Distributed Systems (GRIDS) Laboratory, Department of Computer Science and Software Engineering,The University of Melbourne). Calheiros, R. N., Ranjan, R., De Rose, C. A. F., & Buyya, R. (2009). CloudSim: A novel framework for modeling and simulation of cloud computing infrastructures and services (Tech. Rep., Grid Computing and Distributed Systems (GRIDS) Laboratory, Department of Computer Science and Software Engineering,The University of Melbourne).
Zurück zum Zitat Chen, J., & Yang, Y. (2007). Multiple states based temporal consistency for dynamic verification of fixed-time constraints in grid workflow systems. Concurrency and Computation: Practice and Experience (Wiley), 19(7), 965–982.CrossRef Chen, J., & Yang, Y. (2007). Multiple states based temporal consistency for dynamic verification of fixed-time constraints in grid workflow systems. Concurrency and Computation: Practice and Experience (Wiley), 19(7), 965–982.CrossRef
Zurück zum Zitat Chen, J., & Yang, Y. (2008). A taxonomy of grid workflow verification and validation. Concurrency and Computation: Practice and Experience, 20(4), 347–360.CrossRef Chen, J., & Yang, Y. (2008). A taxonomy of grid workflow verification and validation. Concurrency and Computation: Practice and Experience, 20(4), 347–360.CrossRef
Zurück zum Zitat Chen, J., & Yang, Y. (2010). Temporal dependency based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems. ACM Transactions on Software Engineering and Methodology, to appear. Retrieved 1st February 2010, from http://www.swinflow.org/papers/TOSEM.pdf. Chen, J., & Yang, Y. (2010). Temporal dependency based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems. ACM Transactions on Software Engineering and Methodology, to appear. Retrieved 1st February 2010, from http://​www.​swinflow.​org/​papers/​TOSEM.​pdf.
Zurück zum Zitat Chen, J., & Yang, Y. (2007). Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems. ACM Transactions on Autonomous and Adaptive Systems, 2(2), Article 6.CrossRef Chen, J., & Yang, Y. (2007). Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems. ACM Transactions on Autonomous and Adaptive Systems, 2(2), Article 6.CrossRef
Zurück zum Zitat Chen, J., & Yang, Y. (2008). Temporal dependency based checkpoint selection for dynamic verification of fixed-time constraints in grid workflow systems. Proceedings of the 30th International Conference on Software Engineering (ICSE 2008), Leipzig, Germany, 141–150. Chen, J., & Yang, Y. (2008). Temporal dependency based checkpoint selection for dynamic verification of fixed-time constraints in grid workflow systems. Proceedings of the 30th International Conference on Software Engineering (ICSE 2008), Leipzig, Germany, 141–150.
Zurück zum Zitat Chervenak, A., Deelman, E., Livny, M., Su, M. H., Schuler, R., Bharathi, S., et al. (2007). Data placement for scientific applications in distributed environments. Proceedings of the 8th Grid Computing Conference, 267–274. Chervenak, A., Deelman, E., Livny, M., Su, M. H., Schuler, R., Bharathi, S., et al. (2007). Data placement for scientific applications in distributed environments. Proceedings of the 8th Grid Computing Conference, 267–274.
Zurück zum Zitat Deelman, E., Gannon, D., Shields, M., & Taylor, I. (2008). Workflows and e-Science: An overview of workflow system features and capabilities. Future Generation Computer Systems, 25(6), 528–540. Deelman, E., Gannon, D., Shields, M., & Taylor, I. (2008). Workflows and e-Science: An overview of workflow system features and capabilities. Future Generation Computer Systems, 25(6), 528–540.
Zurück zum Zitat Deelman, E., & Chervenak, A. (2008). Data management challenges of data-intensive scientific workflows. Proceedings of the IEEE International Symposium on Cluster Computing and the Grid, 687–692. Deelman, E., & Chervenak, A. (2008). Data management challenges of data-intensive scientific workflows. Proceedings of the IEEE International Symposium on Cluster Computing and the Grid, 687–692.
Zurück zum Zitat Deelman, E., Singh, G., Livny, M., Berriman, B., & Good, J. (2008). The cost of doing science on the cloud: The montage example. Proceedings of the ACM/IEEE Conference on Supercomputing, Austin, TX, 1–12. Deelman, E., Singh, G., Livny, M., Berriman, B., & Good, J. (2008). The cost of doing science on the cloud: The montage example. Proceedings of the ACM/IEEE Conference on Supercomputing, Austin, TX, 1–12.
Zurück zum Zitat Erl, T. (2008). SOA: Principles of service design. Upper Saddle River, NJ: Prentice Hall. Erl, T. (2008). SOA: Principles of service design. Upper Saddle River, NJ: Prentice Hall.
Zurück zum Zitat Foster, I., Zhao, Y., Raicu, I., & Lu, S. (2008). Cloud computing and grid computing 360-degree compared. Proceedings of the Grid Computing Environments Workshop, 2008, GCE '08, 1–10. Foster, I., Zhao, Y., Raicu, I., & Lu, S. (2008). Cloud computing and grid computing 360-degree compared. Proceedings of the Grid Computing Environments Workshop, 2008, GCE '08, 1–10.
Zurück zum Zitat Hess, A., Holt, J., Jacobson, J., & Seamons, K. E. (2004). Content-triggered trust negotiation. ACM Transactions on Information and System Security, 7(3), 428–456.CrossRef Hess, A., Holt, J., Jacobson, J., & Seamons, K. E. (2004). Content-triggered trust negotiation. ACM Transactions on Information and System Security, 7(3), 428–456.CrossRef
Zurück zum Zitat Hoffa, C., Mehta, G., Freeman, T., Deelman, E., Keahey, K., Berriman, B., et al. (2008). On the use of cloud computing for scientific workflows. Proceedings of the 4th IEEE International Conference on e-Science, 640–645. Hoffa, C., Mehta, G., Freeman, T., Deelman, E., Keahey, K., Berriman, B., et al. (2008). On the use of cloud computing for scientific workflows. Proceedings of the 4th IEEE International Conference on e-Science, 640–645.
Zurück zum Zitat Keahey, K., Figueiredo, R., Fortes, J., Freeman, T., & Tsugawa, M. (2008). Science clouds: Early experiences in cloud computing for scientific applications. Proceedings of the First Workshop on Cloud Computing and its Applications (CCA'08), 1–6. Keahey, K., Figueiredo, R., Fortes, J., Freeman, T., & Tsugawa, M. (2008). Science clouds: Early experiences in cloud computing for scientific applications. Proceedings of the First Workshop on Cloud Computing and its Applications (CCA'08), 1–6.
Zurück zum Zitat Kondo, D., Javadi, B., Malecot, P., Cappello, F., & Anderson, D. P. (2009). Cost-benefit analysis of cloud computing versus desktop grids. Proceedings of the IEEE International Symposium on Parallel & Distributed Processing, IPDPS'09, 1–12. Kondo, D., Javadi, B., Malecot, P., Cappello, F., & Anderson, D. P. (2009). Cost-benefit analysis of cloud computing versus desktop grids. Proceedings of the IEEE International Symposium on Parallel & Distributed Processing, IPDPS'09, 1–12.
Zurück zum Zitat Lin, C., Varadharajan, V., Wang, Y., & Pruthi, V.t. (2004). Enhancing grid security with trust management. Proceedings of the 2004 IEEE International Conference on Services Computing (SCC04), 303–310. Lin, C., Varadharajan, V., Wang, Y., & Pruthi, V.t. (2004). Enhancing grid security with trust management. Proceedings of the 2004 IEEE International Conference on Services Computing (SCC04), 303–310.
Zurück zum Zitat Liu, K., Chen, J. J., Yang, Y., & Jin, H. (2008). A throughput maximization strategy for scheduling transaction-intensive workflows on SwinDeW-G. Concurrency and Computation: Practice and Experience, 20(15), 1807–1820.CrossRef Liu, K., Chen, J. J., Yang, Y., & Jin, H. (2008). A throughput maximization strategy for scheduling transaction-intensive workflows on SwinDeW-G. Concurrency and Computation: Practice and Experience, 20(15), 1807–1820.CrossRef
Zurück zum Zitat Liu, K., Jin, H., Chen, J., Liu, X., Yuan, D., & Yang, Y. (2010). A compromised-time-cost scheduling algorithm in SwinDeW-C for instance-intensive cost-constrained workflows on cloud computing platform. International Journal of High Performance Computing Applications. Liu, K., Jin, H., Chen, J., Liu, X., Yuan, D., & Yang, Y. (2010). A compromised-time-cost scheduling algorithm in SwinDeW-C for instance-intensive cost-constrained workflows on cloud computing platform. International Journal of High Performance Computing Applications.
Zurück zum Zitat Liu, X., Chen, J., Liu, K., & Yang, Y. (2008). Forecasting duration intervals of scientific workflow activities based on time-series patterns. Proceedings of the 4th IEEE International Conference on e-Science (e-Science08), Indianapolis, IN, USA, 23–30. Liu, X., Chen, J., Liu, K., & Yang, Y. (2008). Forecasting duration intervals of scientific workflow activities based on time-series patterns. Proceedings of the 4th IEEE International Conference on e-Science (e-Science08), Indianapolis, IN, USA, 23–30.
Zurück zum Zitat Liu, X., Chen, J., Wu, Z., Ni, Z., Yuan, D., & Yang, Y. (2010). Handling recoverable temporal violations in scientific workflow systems: A workflow rescheduling based strategy. Proceedings of the 10th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid10), Melbourne, Australia. Liu, X., Chen, J., Wu, Z., Ni, Z., Yuan, D., & Yang, Y. (2010). Handling recoverable temporal violations in scientific workflow systems: A workflow rescheduling based strategy. Proceedings of the 10th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid10), Melbourne, Australia.
Zurück zum Zitat Liu, X., Chen, J., & Yang, Y. (September 2008). A probabilistic strategy for setting temporal constraints in scientific workflows. Proceedings of the 6th International Conference on Business Process Management (BPM08), Lecture Notes in Computer Science, Vol. 5240, Milan, Italy, 180–195. Liu, X., Chen, J., & Yang, Y. (September 2008). A probabilistic strategy for setting temporal constraints in scientific workflows. Proceedings of the 6th International Conference on Business Process Management (BPM08), Lecture Notes in Computer Science, Vol. 5240, Milan, Italy, 180–195.
Zurück zum Zitat McCormick, W. T., Sehweitzer, P. J., & White, T. W. (1972). Problem decomposition and data reorganization by a clustering technique. Operations Research, 20, 993–1009.MATHCrossRef McCormick, W. T., Sehweitzer, P. J., & White, T. W. (1972). Problem decomposition and data reorganization by a clustering technique. Operations Research, 20, 993–1009.MATHCrossRef
Zurück zum Zitat Moore, M. (2004). An accurate parallel genetic algorithm to schedule tasks on a cluster. Parallel Computing, 30, 567–583.MathSciNetCrossRef Moore, M. (2004). An accurate parallel genetic algorithm to schedule tasks on a cluster. Parallel Computing, 30, 567–583.MathSciNetCrossRef
Zurück zum Zitat Moretti, C., Bulosan, J., Thain, D., & Flynn, P. J. (2008). All-Pairs: An abstraction for data-intensive cloud computing. Proceedings of the IEEE International Parallel and Distributed Processing Symposium, IPDPS'08, 1–11. Moretti, C., Bulosan, J., Thain, D., & Flynn, P. J. (2008). All-Pairs: An abstraction for data-intensive cloud computing. Proceedings of the IEEE International Parallel and Distributed Processing Symposium, IPDPS'08, 1–11.
Zurück zum Zitat Raghavan, B., Ramabhadran, S., Yocum, K., & Snoeren, A. C. (2007). Cloud control with distributed rate limiting. Proceedings of the 2007 ACM SIGCOMM, Kyoto, Japan, 337–348. Raghavan, B., Ramabhadran, S., Yocum, K., & Snoeren, A. C. (2007). Cloud control with distributed rate limiting. Proceedings of the 2007 ACM SIGCOMM, Kyoto, Japan, 337–348.
Zurück zum Zitat SECES (May 2008). Proceedings of the 1st International Workshop on Software Engineering for Computational Science and Engineering, in conjuction with the 30th International Conference on Software Engineering (ICSE2008), Leipzig, Germany. SECES (May 2008). Proceedings of the 1st International Workshop on Software Engineering for Computational Science and Engineering, in conjuction with the 30th International Conference on Software Engineering (ICSE2008), Leipzig, Germany.
Zurück zum Zitat Simmhan, Y. L., Plale, B., & Gannon, D. (2005). A survey of data provenance in e-Science. SIGMOD Rec. 34(3), 31–36.CrossRef Simmhan, Y. L., Plale, B., & Gannon, D. (2005). A survey of data provenance in e-Science. SIGMOD Rec. 34(3), 31–36.CrossRef
Zurück zum Zitat Wang, L. Z., Jie, W., & Chen, J. (2009). Grid computing: Infrastructure, service, and applications. Boca Raton, FL: CRC Press, Talyor & Francis Group.CrossRef Wang, L. Z., Jie, W., & Chen, J. (2009). Grid computing: Infrastructure, service, and applications. Boca Raton, FL: CRC Press, Talyor & Francis Group.CrossRef
Zurück zum Zitat Weiss, A. (2007). Computing in the cloud. ACM Networker, 11(4), 18–25.CrossRef Weiss, A. (2007). Computing in the cloud. ACM Networker, 11(4), 18–25.CrossRef
Zurück zum Zitat Winsborough, W. H., & Li, N. H. (2006). Safety in automated trust negotiation. ACM Transactions on Information and System Security, 9(3), 352–390.CrossRef Winsborough, W. H., & Li, N. H. (2006). Safety in automated trust negotiation. ACM Transactions on Information and System Security, 9(3), 352–390.CrossRef
Zurück zum Zitat Yang, Y., Liu, K., Chen, J., Lignier, J., & Jin, H. (December 2007). Peer-to-peer based grid workflow runtime environment of swinDeW-G. Proceedings of the 3rd International Conference on e-Science and Grid Computing (e-Science07), Bangalore, India, 51–58. Yang, Y., Liu, K., Chen, J., Lignier, J., & Jin, H. (December 2007). Peer-to-peer based grid workflow runtime environment of swinDeW-G. Proceedings of the 3rd International Conference on e-Science and Grid Computing (e-Science07), Bangalore, India, 51–58.
Zurück zum Zitat Yang, Y., Liu, K., Chen, J., Liu, X., Yuan, D., & Jin, H. (December 2008). An algorithm in swinDeW-C for scheduling transaction-intensive cost-constrained cloud workflows. Proceedings of the 4th IEEE International Conference on e-Science (e-Science08), Indianapolis, IN, USA, 374–375. Yang, Y., Liu, K., Chen, J., Liu, X., Yuan, D., & Jin, H. (December 2008). An algorithm in swinDeW-C for scheduling transaction-intensive cost-constrained cloud workflows. Proceedings of the 4th IEEE International Conference on e-Science (e-Science08), Indianapolis, IN, USA, 374–375.
Zurück zum Zitat Yu, J., & Buyya, R. (2005). A taxonomy of workflow management systems for grid computing. Journal of Grid Computing, (3), 171–200. Yu, J., & Buyya, R. (2005). A taxonomy of workflow management systems for grid computing. Journal of Grid Computing, (3), 171–200.
Metadaten
Titel
SwinDeW-C: A Peer-to-Peer Based Cloud Workflow System
verfasst von
Xiao Liu
Dong Yuan
Gaofeng Zhang
Jinjun Chen
Yun Yang
Copyright-Jahr
2010
Verlag
Springer US
DOI
https://doi.org/10.1007/978-1-4419-6524-0_13