Skip to main content
Erschienen in: Cluster Computing 3/2015

01.09.2015

VM auto-scaling methods for high throughput computing on hybrid infrastructure

verfasst von: Jieun Choi, Younsun Ahn, Seoyoung Kim, Yoonhee Kim, Jaeyoung Choi

Erschienen in: Cluster Computing | Ausgabe 3/2015

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Cloud computing provides on-demand resource provisioning and scalable resources dynamically for the efficient use of computing resources. Scientific applications recently need a very large number of loosely coupled tasks to be handled efficiently. In response, current computing environments often consist of heterogeneous resources such as cloud computing. To effectively use cloud resources, auto-scaling methods that consider diverse metrics such as CPU utilization and costs of resource usage have been studied widely. However it still remains a challenge to automatically and timely allocate resources such that deadline violation and application types are considered. In this paper, we propose auto-scaling methods that consider specific conditions such as application types, task dependency, user-defined deadlines and data transfer times within a hybrid computing infrastructure. Our hybrid computing infrastructure consists of local cluster and cloud resources using HTCaaS. We observe noticeable improvements in performance when our auto-scaling methods for bag-of-tasks and workflow applications is applied.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Cirne, W., Brasileiro, F., Sauve, J., Andrade, N., Paranhos, D., Santos-Neto, E., Medeiros, R.: Grid computing for bag of jobs applications. In: Proceedings of the 3rd IFIP Conference on E-Commerce, E-Business and E-Government, 21–23 Sept 2003 Cirne, W., Brasileiro, F., Sauve, J., Andrade, N., Paranhos, D., Santos-Neto, E., Medeiros, R.: Grid computing for bag of jobs applications. In: Proceedings of the 3rd IFIP Conference on E-Commerce, E-Business and E-Government, 21–23 Sept 2003
2.
Zurück zum Zitat O’Brien, A., Newhouse, S., Darlington, J.: Mapping of Scientific Workflow Within the E-Protein project to Distributed Resources. In: UK E-Sceince All Hands Meeting, Nottingham (2004) O’Brien, A., Newhouse, S., Darlington, J.: Mapping of Scientific Workflow Within the E-Protein project to Distributed Resources. In: UK E-Sceince All Hands Meeting, Nottingham (2004)
3.
Zurück zum Zitat Kang, H., Koh, J., Kim, Y.: “A SLA driven VM auto-scaling method in hybrid cloud environment. In: Network Operations and Management Symposium (APNOMS), 2013 15th Asia-Pacific, Hiroshima, Japan, 25–28 Sept 2013 Kang, H., Koh, J., Kim, Y.: “A SLA driven VM auto-scaling method in hybrid cloud environment. In: Network Operations and Management Symposium (APNOMS), 2013 15th Asia-Pacific, Hiroshima, Japan, 25–28 Sept 2013
5.
Zurück zum Zitat Liu, C.-Y., Shie, M.-R., Lee, Y.-F., Lin, Y.-C., Lai, K.-C.: Vertical/horizontal resource scaling mechanism for federated clouds (2014) Liu, C.-Y., Shie, M.-R., Lee, Y.-F., Lin, Y.-C., Lai, K.-C.: Vertical/horizontal resource scaling mechanism for federated clouds (2014)
6.
Zurück zum Zitat Lorido-Bortran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. J. Grid Comput. 12(4), 559–592 (2014)CrossRef Lorido-Bortran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. J. Grid Comput. 12(4), 559–592 (2014)CrossRef
7.
Zurück zum Zitat Bao, J., Lu, Z., Wu, J., Zhang, S., Zhong, Y.: Implementing a novel load-aware auto scale scheme for private cloud resource management platform. In: Network Operations and Management Symposium (NOMS) (2014) Bao, J., Lu, Z., Wu, J., Zhang, S., Zhong, Y.: Implementing a novel load-aware auto scale scheme for private cloud resource management platform. In: Network Operations and Management Symposium (NOMS) (2014)
8.
Zurück zum Zitat Yang, J., Liu, C., Shang, Y., Cheng, B., Mao Z., et al.: A cost-aware auto-scaling approach using the workload prediction in service clouds. In: 6th IEEE International Conference on Cloud Computing (CLOUD), pp. 810–815 (2013) Yang, J., Liu, C., Shang, Y., Cheng, B., Mao Z., et al.: A cost-aware auto-scaling approach using the workload prediction in service clouds. In: 6th IEEE International Conference on Cloud Computing (CLOUD), pp. 810–815 (2013)
9.
Zurück zum Zitat Saleh, O., GropengieBer, F., Betz, H., Mandarawi W., Sattler, K.: Monitoring and auto-scaling iaas clouds: a case for complex event processing on data streams. In: 6th IEEE/ACM International Conference on Utility and Cloud Computing, pp. 387–392 (2013) Saleh, O., GropengieBer, F., Betz, H., Mandarawi W., Sattler, K.: Monitoring and auto-scaling iaas clouds: a case for complex event processing on data streams. In: 6th IEEE/ACM International Conference on Utility and Cloud Computing, pp. 387–392 (2013)
10.
Zurück zum Zitat Dutta, S., Gera, S., Vermam A., Viswanathan, B.: Smartscale: automatic application scaling in enterprise cloud. In: 5th IEEE International Conference on Cloud Comuting (CLOUD), pp. 221–228 (2012) Dutta, S., Gera, S., Vermam A., Viswanathan, B.: Smartscale: automatic application scaling in enterprise cloud. In: 5th IEEE International Conference on Cloud Comuting (CLOUD), pp. 221–228 (2012)
11.
Zurück zum Zitat Mao, M., Humphrey, M.: Scaling and Scheduling to Maximize Application Performance within Budget Constraints in Cloud Workflows. In: IEEE 27th International Symposium on Parallel and Distributed Processing (2013) Mao, M., Humphrey, M.: Scaling and Scheduling to Maximize Application Performance within Budget Constraints in Cloud Workflows. In: IEEE 27th International Symposium on Parallel and Distributed Processing (2013)
12.
Zurück zum Zitat Yu, J., Buyya R., Tham, C.K.: Cost-based scheduling of scientific workflow applications on utility grids. In: 1st IEEE International Conference on E-Science and Grid Computing, Melbourne, 5–8 Dec 2005 Yu, J., Buyya R., Tham, C.K.: Cost-based scheduling of scientific workflow applications on utility grids. In: 1st IEEE International Conference on E-Science and Grid Computing, Melbourne, 5–8 Dec 2005
13.
Zurück zum Zitat Abrishami, S., Naghibzadeh, M., Epema, D.H.: Deadline-donstrained workflow scheduling algorithms for infrastructure as a service clouds. J. Future Gener. Comput. Syst. 29(1), 158–169 (2013)CrossRef Abrishami, S., Naghibzadeh, M., Epema, D.H.: Deadline-donstrained workflow scheduling algorithms for infrastructure as a service clouds. J. Future Gener. Comput. Syst. 29(1), 158–169 (2013)CrossRef
14.
Zurück zum Zitat Bittencourt, L.F., Madeira, E.R.: A performance oriented adaptive scheduler for dependent tasks on grids. Concurr. Comput. 20(9), 1029–1049 (2008)CrossRef Bittencourt, L.F., Madeira, E.R.: A performance oriented adaptive scheduler for dependent tasks on grids. Concurr. Comput. 20(9), 1029–1049 (2008)CrossRef
15.
Zurück zum Zitat Rizos, S., et al.: Integrated Research in GRID Computing. Scheduling workflows with budget constraints. Springer, Berlin (2007) Rizos, S., et al.: Integrated Research in GRID Computing. Scheduling workflows with budget constraints. Springer, Berlin (2007)
16.
Zurück zum Zitat Niu, S., et al.: Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters. In: Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, p. 56 (2013) Niu, S., et al.: Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters. In: Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, p. 56 (2013)
20.
Zurück zum Zitat Johnson, M., Zaretskaya, I., Raytselis, Y., Merezhuk, Y., McGinnis, S., Maddent, T.L.: NCBI BLAST: a better web interface. Nucl. Acids Res. 36, W5–W9 (2008)CrossRef Johnson, M., Zaretskaya, I., Raytselis, Y., Merezhuk, Y., McGinnis, S., Maddent, T.L.: NCBI BLAST: a better web interface. Nucl. Acids Res. 36, W5–W9 (2008)CrossRef
21.
Zurück zum Zitat Bergman, N.H., Bhagwat, M., Aravind, L.: PSI-BLAST Tutorial (2007) Bergman, N.H., Bhagwat, M., Aravind, L.: PSI-BLAST Tutorial (2007)
23.
Zurück zum Zitat Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A.F., Buyya, R.: Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41(1), 23–50 (2011)CrossRef Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A.F., Buyya, R.: Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41(1), 23–50 (2011)CrossRef
Metadaten
Titel
VM auto-scaling methods for high throughput computing on hybrid infrastructure
verfasst von
Jieun Choi
Younsun Ahn
Seoyoung Kim
Yoonhee Kim
Jaeyoung Choi
Publikationsdatum
01.09.2015
Verlag
Springer US
Erschienen in
Cluster Computing / Ausgabe 3/2015
Print ISSN: 1386-7857
Elektronische ISSN: 1573-7543
DOI
https://doi.org/10.1007/s10586-015-0462-8

Weitere Artikel der Ausgabe 3/2015

Cluster Computing 3/2015 Zur Ausgabe