nach oben

Erschienen in:

2017 | OriginalPaper | Buchkapitel

A Profit-Maximum Resource Allocation Approach for Mapreduce in Data Centers

verfasst von : Xiaolu Zhang, Weidong Li, Xi Liu, Xuejie Zhang

Erschienen in: Green, Pervasive, and Cloud Computing

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Resource allocation for Mapreduce data processing poses difficult challenges to system administrators in data centers. The extreme scale of Mapreduce applications require an efficiently profitable resource allocation algorithm that minimizes the energy consumption cost while maintaining the highest level of performance. In this paper, we propose a profit-maximum model that minimizes the cost of energy consumption and makespan. By adopting a minimum-weight b-matching rounding algorithm (MBRA) to find an integer solution, then assign map/reduce tasks to individual slots to build a complete resource allocation. Finally, we perform experiments on real workload to evaluate the profit-maximum model and analyze the performance of our proposed algorithm. The results show that MBRA is able to find a near-optimal integer solution that maximizes the profit per unit time in a lower runtime, and it is up to 30%~70% in profit that is better than the current heuristic scheduling algorithm and the rounding algorithm.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel CDLP: A Core Distributing Policy Based on Logic Partitioning

Nächstes Kapitel An Efficient Implementation of the Algorithm by Lukáš et al. on Hadoop

Moseley, B., Dasgupta, A., Kumar, R., Sarlos, T.: On scheduling in map-reduce and flow-shops. In: Proceedings of the 23rd Annual ACM Symposium on Parallelism in Algorithms and Architectures, pp. 289–298 (2011)

Tang, S., Lee, B.S., He, B.: Dynamic job ordering and slot configurations for mapreduce workloads. IEEE Trans. Serv. Comput. 9(1), 4–17 (2016)CrossRef

Zhu, Y., Jiang, Y., Wu, W., Ding, L.: Minimizing makespan and total completion time in mapreduce-like systems. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications (INFOCOM 2014), pp. 2166–2174 (2014)

Kolb, L., Thor, A., Rahm, E.: Load balancing for mapreduce-based entity resolution. In: Proceedings of the 28th IEEE International Conference on Data Engineering, pp. 618–629 (2012)

Wang, W., Zhu, K., Ying, L., Tan, J., Zhang, L.: MapTask scheduling in mapreduce with data locality: throughput and heavy-traffic optimality. IEEE/ACM Trans. Netw. 24(1), 190–203 (2016)CrossRef

Wang, X., Wang, Y., Zhu, H.: Energy-efficient task scheduling model based on MapReduce for cloud computing using genetic algorithm. J. Comput. 7(12), 2962–2970 (2012)CrossRef

Mashayekhy, L., Nejad, M.M., Grosu, D., Zhang, Q., Shi, W.: Energy-aware scheduling of mapreduce jobs for big data applications. IEEE Trans. Parallel Distrib. Syst. 26(10), 2720–2733 (2015)CrossRef

Tarplee, K.M., Maciejewski, A.A., Siegel, H.J.: Energy-aware profit maximizing scheduling algorithm for heterogeneous computing systems. In: Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014), pp. 595–603 (2014)

Ren, Z.J., Wan, J., Shi, W.S., Xu, X.H., Zhou, M.: Workload analysis, implications, and optimization on a production hadoop cluster: a case study on taobao. IEEE Trans. Serv. Comput. 7(2), 307–321 (2014)CrossRef

10.

Verma, A., Cherkasova, L., Campbell, R.H.: Two sides of a coin: optimizing the schedule of mapreduce jobs to minimize their makespan and improve cluster performance. In: Proceedings of the 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, pp. 11–18 (2012)

11.

Song, J., Wang, Z., Li, T.T., Yu, G.: Energy consumption optimization data placement algorithm for mapreduce system. J. softw. 26(8), 2091–2110 (2015)MathSciNet

12.

Lin, B., Li, S.S., Liao, X.K., Meng, L.B., Liu, X.D., Huang, H.: Seadown: SLA-aware size-scaling power management in heterogeneous mapreduce cluster. J. Comput. 36(5), 977–987 (2013)

13.

Maheshwari, N., Nanduri, R., Varma, V.: Dynamic energy efficient data placement and cluster reconfiguration algorithm for mapreduce framework. Future Gener. Comput. Syst. 28(1), 119–127 (2012)CrossRef

14.

Lin, J.C., Leu, F.Y., Chen, Y.: Impact of mapreduce policies on job completion reliability and job energy consumption. IEEE Trans. Parallel Distrib. Syst. 26(5), 1364–1378 (2015)CrossRef

15.

Tian, F., Chen, K.: Towards optimal resource provisioning for running mapreduce programs in public clouds. In: Proceedings of 2011 IEEE International Conference on Cloud Computing (CLOUD 2011), pp. 155–162 (2011)

16.

Palanisamy, B., Singh, A., Liu, L.: Cost-effective resource provisioning for mapreduce in a cloud. IEEE Trans. Parallel Distrb. Syst. 26(5), 1265–1279 (2015)CrossRef

17.

Young, B.D., Apodaca, J., Briceño, L.D., Smith, J., Pasricha, S., Maciejewski, A.A., Siegel, H.J., Khemka, B., Bahirat, S., Ramirez, A., Zou, Y.: Deadline and energy constrained dynamic resource allocation in a heterogeneous computing environment. J. Supercomput. 63(2), 326–347 (2013)CrossRef

18.

Li, W.D., Liu, X., Zhang, X.J., Cai, X.B.: A Task-type-based algorithm for the energy-aware profit maximizing scheduling problem in heterogeneous computing systems. In: Proceedings of the 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2015), pp. 1107–1110 (2015)

19.

Jansen, K., Porkolab, L.: Improved approximation schemes for scheduling unrelated parallel machines. Math. Oper. Res. 26(2), 324–338 (2001)MathSciNetCrossRefMATH

20.

Shmoys, D.B., Tardos, É.: An approximation algorithm for the generalized assignment problem. Math. Prog. 62(1–3), 461–474 (1993)MathSciNetCrossRefMATH

21.

Huang, B.C., Jebara, T.: Fast b-matching via sufficient selection belief propagation. In: Proceedings of AISTATS, pp. 361–369 (2011)

Titel: A Profit-Maximum Resource Allocation Approach for Mapreduce in Data Centers
verfasst von: Xiaolu Zhang
Weidong Li
Xi Liu
Xuejie Zhang
Verlag: Springer International Publishing
Buch: Green, Pervasive, and Cloud Computing
Print ISBN: 978-3-319-57185-0

Electronic ISBN: 978-3-319-57186-7

Copyright-Jahr: 2017
DOI: https://doi.org/10.1007/978-3-319-57186-7_34

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"