nach oben

Erschienen in:

2018 | OriginalPaper | Buchkapitel

Pricing Cloud Resource Based on Reinforcement Learning in the Competing Environment

verfasst von : Bing Shi, Hangxing Zhu, Han Yuan, Rongjian Shi, Jinwen Wang

Erschienen in: Cloud Computing – CLOUD 2018

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Multiple cloud providers compete against each other in order to attract cloud users and make profits in the cloud market. In doing so, each provider needs to charge fees to users in a proper way. In this paper, we will analyze how a cloud provider sets price effectively when competing against other cloud providers. Specifically, we model this problem as a Markov game, and then use minimax-Q and Q learning algorithms to design the pricing policies respectively. Based on this, we run extensive experiments to analyze the effectiveness of minimax-Q and Q learning based pricing policies. We find that although minimax-Q is more suitable in analyzing the competing game with multiple self-interested cloud providers, Q learning based pricing policy performs better in terms of making profits. We also find that minimax-Q learning based pricing policy performs better in terms of keeping cloud users. Our experimental results can provide useful insights on designing practical pricing policies in different situations.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Renewable Energy Curtailment via Incentivized Inter-datacenter Workload Migration

Nächstes Kapitel An Effective Offloading Trade-Off to Economize Energy and Performance in Edge Computing

Our model can be easily extended to the case with more than two cloud providers.

minimax-Q learning was designed to solve Markov game when its stage game is a zero-sum game. In this paper, although the sum of both providers’ payoffs is not zero, the gain of one provider (users choosing this provider) is indeed the loss of the other provider (users not choosing that provider). Therefore, it is actually a zero-sum game, and we use minimax-Q learning algorithm to design the pricing policy in this competing environment.

Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., Zaharia, M.: A view of cloud computing: clearing the clouds away from the true potential and obstacles posed by this computing capability. Commun. ACM 53(4), 50–58 (2010)CrossRef

Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandicl, I.: Cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6), 599–616 (2009)CrossRef

Foster, I., Zhao, Y., Raicu, I., Lu, S.: Cloud computing and grid computing 360-degree compared. In: Grid Computing Environments Workshop, pp. 1–10 (2008)

Buyya, R., Yeo, C., Venugopal, S.: Market-oriented cloud computing: vision, hype, and reality for delivering it services as computing utilities. In: The 10th IEEE International Conference on High Performance Computing and Communications, pp. 5–13 (2008)

Laatikainen, G., Ojala, A., Mazhelis, O.: Cloud services pricing models. In: Herzwurm, G., Margaria, T. (eds.) ICSOB 2013. LNBIP, vol. 150, pp. 117–129. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39336-5_12CrossRef

Sharma, B., Thulasiram, R.K., Thulasiraman, P., Garg, S.K., Buyya, R.: Pricing cloud compute commodities: a novel financial economic model. In: The 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 451–457 (2012)

Wang, H., Jing, Q., Chen, R., He, B., Qian, Z., Zhou, L.: Distributed systems meet economics: pricing in the cloud. In: The 2nd USENIX Conference on Hot Topics in Cloud Computing, pp. 1–6 (2010)

Feng, Y., Li, B., Li, B.: Price competition in an oligopoly market with multiple iaas cloud providers. IEEE Trans. Comput. 63(1), 59–73 (2014)MathSciNetCrossRef

Kantere, V., Dash, D., Francois, G., Kyriakopoulou, S., Ailamaki, A.: Optimal service pricing for a cloud cache. IEEE Trans. Knowl. Data Eng. 23(9), 1345–1358 (2011)CrossRef

10.

Xu, H., Li, B.: Maximizing revenue with dynamic cloud pricing: the infinite horizon case. In: IEEE International Conference on Communications, pp. 2929–2933 (2012)

11.

Vengerov, D.: A gradient-based reinforcement learning approach to dynamic pricing in partially-observable environments. Future Gener. Comput. Syst. 24(7), 687–693 (2008)CrossRef

12.

Xu, H., Li, B.: Dynamic cloud pricing for revenue maximization. IEEE Trans. Cloud Comput. 1(2), 158–171 (2013)MathSciNetCrossRef

13.

Xu, B., Qin, T., Qiu, G., Liu, T.Y.: Optimal pricing for the competitive and evolutionary cloud market. In: The 24th International Joint Conference on Artificial Intelligence, pp. 139–145 (2015)

14.

Truong-Huu, T., Tham, C.K.: A game-theoretic model for dynamic pricing and competition among cloud providers. In: The 6th International Conference on Utility and Cloud Computing, pp. 235–238 (2013)

15.

Truong-Huu, T., Tham, C.K.: A novel model for competition and cooperation among cloud providers. IEEE Trans. Cloud Comput. 2(3), 251–265 (2014)CrossRef

16.

Wal, J.: Stochastic dynamic programming. Methematisch Centrum (1980)

17.

Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: 11th International Conference on Machine Learning, pp. 157–163 (1994)CrossRef

18.

Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)MATH

19.

Adams, I.F., Long, D.D.E., Miller, E.L., Pasupathy, S., Storer, M.W.: Maximizing efficiency by trading storage for computation. In: The 1st USENIX Conference on Hot Topics in Cloud Computing. vol. 7 (2009)

20.

Jung, H., Klein, C.M.: Optimal inventory policies under decreasing cost functions via geometric programming. Eur. J. Oper. Res. 132(3), 628–642 (2001)CrossRef

21.

Train, K.E.: Discrete Choice Methods with Simulation. Cambridge University Press, Cambridge (2003)CrossRef

Titel: Pricing Cloud Resource Based on Reinforcement Learning in the Competing Environment
verfasst von: Bing Shi
Hangxing Zhu
Han Yuan
Rongjian Shi
Jinwen Wang
Verlag: Springer International Publishing
Buch: Cloud Computing – CLOUD 2018
Print ISBN: 978-3-319-94294-0

Electronic ISBN: 978-3-319-94295-7

Copyright-Jahr: 2018
DOI: https://doi.org/10.1007/978-3-319-94295-7_11

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner