nach oben

Computing

Erschienen in:

29.01.2022 | Regular Paper

Virtual machine migration policy for multi-tier application in cloud computing based on Q-learning algorithm

verfasst von: Cong Hung Tran, Thanh Khiet Bui, Tran Vu Pham

Erschienen in: Computing | Ausgabe 6/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Cloud computing technology provides shared computing which can be accessed over the Internet. When cloud data centers are flooded by end-users, how to efficiently manage virtual machines to balance both economical cost and ensure QoS becomes a mandatory work to service providers. Virtual machine migration feature brings a plenty of benefits to stakeholders such as cost, energy, performance, stability, availability. However, stakeholders’ objectives are usually conflict with each other. Furthermore, the optimal resource allocation problem in cloud infrastructure is usually NP-Hard or NP-Complete class. In this paper, the virtual migration problem is formulated by applying the game theory to ensure both load balance and resource utilization. The virtual machine migration algorithm, named V2PQL, is proposed based on Markov decision process and Q-learning algorithm. The results of the simulation demonstrate the efficiency of our proposal which are divided into training phase and extraction phase. The proposed V2PQL algorithm has been benchmarked to the Round-Robin, inverse Ant System, Max–Min Ant System, and Ant System algorithms in order to highlight its strength and feasibility in extraction phase.

Vorheriger Artikel Solving dimension reduction problems for classification using Promoted Crow Search Algorithm (PCSA)

Nächster Artikel An efficient gray system theory-based routing protocol for energy consumption management in the Internet of Things using fog and cloud computing

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Bai WH, Xi JQ, Zhu JX, Huang SW (2015) Performance analysis of heterogeneous data centers in cloud computing using a complex queuing model. Math Probl Eng. https://doi.org/10.1155/2015/980945MathSciNetCrossRefMATH

Baker T, Mackay M, Randles M, Taleb-Bendiab A (2013) Intention-oriented programming support for runtime adaptive autonomic cloud-based applications. Comput Electr Eng 39(7):2400–2412. https://doi.org/10.1016/j.compeleceng.2013.04.019CrossRef

Bui KT, Ho HD, Pham TV, Tran HC (2020) Virtual machines migration game approach for multi-tier application in infrastructure as a service cloud computing. IET Netw 9(6):326–337. https://doi.org/10.1049/iet-net.2019.0204CrossRef

Bui KT, Nguyen LV, Tran TV, Pham TV, Tran HC (2021) A load balancing vms migration approach for multi-tier application in cloud computing based on fuzzy set and q-learning algorithm. In: Research in intelligent and computing in engineering. Springer, pp 617–628. https://doi.org/10.1007/978-981-15-7527-3_58

Bui KT, Pham TV, Tran HC (2016) A load balancing game approach for vm provision cloud computing based on ant colony optimization. In: International conference on context-aware systems and applications. Springer, pp 52–63. https://doi.org/10.1007/978-3-319-56357-2_6

Duong T, Chu YJ, Nguyen T, Chakareski J (2015) Virtual machine placement via q-learning with function approximation. In: 2015 IEEE global communications conference (GLOBECOM), pp 1–6. IEEE. https://doi.org/10.1109/GLOCOM.2015.7417491

Farahnakian F, Liljeberg P, Plosila J (2014) Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In: 2014 22nd Euromicro international conference on parallel, distributed, and network-based processing. IEEE, pp 500–507. https://doi.org/10.1109/PDP.2014.109

Ficco M, Esposito C, Palmieri F, Castiglione A (2018) A coral-reefs and game theory-based approach for optimizing elastic cloud resource allocation. Fut Gener Comput Syst 78:343–352. https://doi.org/10.1016/j.future.2016.05.025CrossRef

Fujiwara-Greve T (1989) Learning from delayed rewards, vol 1. King’s College, Cambridge

10.

Fujiwara-Greve T (2015) Non-cooperative game theory, vol 1. Springer, BerlinCrossRef

11.

Gao Y, Guan H, Qi Z, Hou Y, Liu L (2013) A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. J Comput Syst Sci 79(8):1230–1242. https://doi.org/10.1016/j.jcss.2013.02.004MathSciNetCrossRefMATH

12.

Ghasemi A, Toroghi Haghighat A (2020) A multi-objective load balancing algorithm for virtual machine placement in cloud data centers based on machine learning. Computing 102:2049–2072. https://doi.org/10.1007/s00607-020-00813-wMathSciNetCrossRef

13.

Ghumman NS, Kaur R (2015) Dynamic combination of improved max-min and ant colony algorithm for load balancing in cloud system. In: 2015 6th International conference on computing, communication and networking technologies (ICCCNT). IEEE, pp 1–5. https://doi.org/10.1109/ICCCNT.2015.7395172

14.

Guo Y, Stolyar A, Walid A (2018) Online vm auto-scaling algorithms for application hosting in a cloud. IEEE Trans Cloud Comput. https://doi.org/10.1109/TCC.2018.2830793CrossRef

15.

Hartmanis J (1982) Computers and intractability: a guide to the theory of np-completeness. SIAM Rev 24(1):90. https://doi.org/10.1137/1024022CrossRef

16.

Hsieh SY, Liu CS, Buyya R, Zomaya AY (2020) Utilization-prediction-aware virtual machine consolidation approach for energy-efficient cloud data centers. J Parallel Distrib Comput 139:99–109. https://doi.org/10.1016/j.jpdc.2019.12.014CrossRef

17.

Huang G, Wang S, Zhang M, Li Y, Qian Z, Chen Y, Zhang S (2016) Auto scaling virtual machines for web applications with queueing theory. In: 2016 3rd International conference on systems and informatics (ICSAI). IEEE, pp 433–438. https://doi.org/10.1109/ICSAI.2016.7810994

18.

Jamshidi P, Sharifloo AM, Pahl C, Metzger A, Estrada G (2015) Self-learning cloud controllers: fuzzy q-learning for knowledge evolution. In: 2015 International conference on cloud and autonomic computing. IEEE, pp 208–211. https://doi.org/10.1109/ICCAC.2015.35

19.

Levin E, Pieraccini R, Eckert W (1998) Using Markov decision process for learning dialogue strategies. In: Proceedings of the 1998 IEEE international conference on acoustics, speech and signal processing (ICASSP’98) (Cat. No. 98CH36181), vol 1. IEEE, pp 201–204. https://doi.org/10.1109/ICASSP.1998.674402

20.

Minarolli D, Freisleben B, (2011) Utility-based resource allocation for virtual machines in cloud computing. In: 2011 IEEE symposium on computers and communications (ISCC). IEEE, pp 410–417. https://doi.org/10.1109/ISCC.2011.5983872

21.

Morton T, Pentico DW (1993) Heuristic scheduling systems: with applications to production systems and project management, vol 3. Wiley

22.

Noshy M, Ibrahim A, Ali HA (2018) Optimization of live virtual machine migration in cloud computing: a survey and future directions. J Netw Comput Appl 110:1–10. https://doi.org/10.1016/j.jnca.2018.03.002CrossRef

23.

Rolik O, Zharikov E, Koval A, Telenyk S (2018) Dynamie management of data center resources using reinforcement learning. In: 2018 14th International conference on advanced trends in radioelecrtronics, telecommunications and computer engineering (TCSET). IEEE, pp 237–244. https://doi.org/10.1109/TCSET.2018.8336194

24.

Rybina K, Schill A (2016) Estimating energy consumption during live migration of virtual machines. In: 2016 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), pp 1–5. IEEE. https://doi.org/10.1109/BlackSeaCom.2016.7901567

25.

Sahni J, Vidyarthi DP (2017) Heterogeneity-aware adaptive auto-scaling heuristic for improved qos and resource usage in cloud environments. Computing 99(4):351–381. https://doi.org/10.1007/s00607-016-0530-9MathSciNetCrossRef

26.

Saovapakhiran B, Michailidis G, Devetsikiotis M, (2011) Aggregated-dag scheduling for job flow maximization in heterogeneous cloud computing. In: 2011 IEEE global telecommunications conference-GLOBECOM 2011. IEEE, pp 1–6. https://doi.org/10.1109/GLOCOM.2011.6133611

27.

Siar H, Kiani K, Chronopoulos AT (2015) An effective game theoretic static load balancing applied to distributed computing. Clust Comput 18(4):1609–1623. https://doi.org/10.1007/s10586-015-0486-0CrossRef

28.

Tsai CW, Rodrigues JJ (2013) Metaheuristic scheduling for cloud: a survey. IEEE Syst J 8(1):279–291. https://doi.org/10.1109/JSYST.2013.2256731CrossRef

29.

Van Laarhoven PJ, Aarts EH, Lenstra JK (1992) Job shop scheduling by simulated annealing. Oper Res 40(1):113–125. https://doi.org/10.1287/opre.40.1.113MathSciNetCrossRefMATH

30.

Van Otterlo M, Wiering M (2012) Reinforcement learning and markov decision processes. In: Reinforcement learning. Springer, pp 3–42. https://doi.org/10.1007/978-3-642-27645-3_1

31.

Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292MATH

32.

Xiao Z, Song W, Chen Q (2012) Dynamic resource allocation using virtual machines for cloud computing environment. IEEE Trans Parallel Distrib Syst 24(6):1107–1117. https://doi.org/10.1109/TPDS.2012.283CrossRef

33.

Xu X, Yu H (2014) A game theory approach to fair and efficient resource allocation in cloud computing. Math Probl Eng. https://doi.org/10.1155/2014/915878MathSciNetCrossRefMATH

34.

Yang L, Feng Y, Li K (2017) Optimization of virtual resources provisioning for cloud applications to cope with traffic burst. In: 2017 IEEE international symposium on parallel and distributed processing with applications and 2017 IEEE international conference on ubiquitous computing and communications (ISPA/IUCC). IEEE, pp 80–87. https://doi.org/10.1109/ISPA/IUCC.2017.00021

35.

Ye D, Chen J (2013) Non-cooperative games on multidimensional resource allocation. Fut Gener Comput Syst 29(6):1345–1352. https://doi.org/10.1016/j.future.2013.02.004CrossRef

36.

Zhang Q, Cheng L, Boutaba R (2010) Cloud computing: state-of-the-art and research challenges. J Internet Serv Appl 1(1):7–18. https://doi.org/10.1007/s13174-010-0007-6CrossRef

Titel: Virtual machine migration policy for multi-tier application in cloud computing based on Q-learning algorithm
verfasst von: Cong Hung Tran
Thanh Khiet Bui
Tran Vu Pham
Publikationsdatum: 29.01.2022
Verlag: Springer Vienna
Erschienen in: Computing / Ausgabe 6/2022
Print ISSN: 0010-485X
Elektronische ISSN: 1436-5057
DOI: https://doi.org/10.1007/s00607-021-01047-0

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 6/2022

Users opinion and emotion understanding in social media regarding COVID-19 vaccine

Evaluation of fog application placement algorithms: a survey

Scalable blockchain storage systems: research progress and models

Solving dimension reduction problems for classification using Promoted Crow Search Algorithm (PCSA)

Network structure optimization for social networks by minimizing the average path length

Trust on wheels: Towards secure and resource efficient IoV networks