Skip to main content
Erschienen in: Journal of Network and Systems Management 4/2022

01.10.2022

An Efficient and Decentralized Fuzzy Reinforcement Learning Bandwidth Controller for Multitenant Data Centers

verfasst von: Reiner H. Santos Filho, Tadeu N. Ferreira, Diogo M. F. Mattos, Dianne S. V. Medeiros

Erschienen in: Journal of Network and Systems Management | Ausgabe 4/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Cloud service providers rely on bandwidth overprovisioning to avoid Service Level Agreements’ violation (SLAs) when allocating tenants’ resources in multitenant cloud environments. Tenants’ network usage is usually dynamic, but the shared resources are often allocated statically and in batches, causing resource idleness. This paper envisions an opportunity for optimizing cloud service networks. As such, we propose an autonomous bandwidth allocation mechanism based on Fuzzy Reinforcement Learning (FRL) to reduce the idleness of cloud network resources. Our mechanism dynamically allocates resources, prioritizing tenants and allowing them to exceed the contracted bandwidth temporarily without violating the SLAs. We assess our mechanism by comparing FRL usage against pure Fuzzy Inference System (FIS) and pure Reinforcement Learning (RL). The evaluation scenario is an emulation in which tenants share resources from a cloud provider and generate traffic based on real HTTP traffic. The results show that our mechanism increases tenant’s cloud network utilization by 30% compared to FIS while maintaining the cloud traffic load within a healthy threshold and more stable than RL.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
Literatur
1.
Zurück zum Zitat Shen, H., Li, Z.: New bandwidth sharing and pricing policies to achieve a win-win situation for cloud provider and tenants. IEEE Trans. Parallel Distrib. Syst. 27(9), 2682–2697 (2015)CrossRef Shen, H., Li, Z.: New bandwidth sharing and pricing policies to achieve a win-win situation for cloud provider and tenants. IEEE Trans. Parallel Distrib. Syst. 27(9), 2682–2697 (2015)CrossRef
2.
Zurück zum Zitat Malbašić, T., Bojović, P.D., Bojović, Ž, Šuh, J., Vujošević, D.: Hybrid SDN networks: a multi-parameter server load balancing scheme. J. Netw. Syst. Manag. 30(2), 30 (2022)CrossRef Malbašić, T., Bojović, P.D., Bojović, Ž, Šuh, J., Vujošević, D.: Hybrid SDN networks: a multi-parameter server load balancing scheme. J. Netw. Syst. Manag. 30(2), 30 (2022)CrossRef
3.
Zurück zum Zitat Son, J., Buyya, R.: Priority-aware VM allocation and network bandwidth provisioning in software-defined networking (SDN)-enabled clouds. IEEE Trans. Sustain. Comput. 4(1), 17–28 (2018)CrossRef Son, J., Buyya, R.: Priority-aware VM allocation and network bandwidth provisioning in software-defined networking (SDN)-enabled clouds. IEEE Trans. Sustain. Comput. 4(1), 17–28 (2018)CrossRef
4.
Zurück zum Zitat Nine, M.S.Z., Azad, M.A.K., Abdullah, S., Rahman, R.M.: Fuzzy logic based dynamic load balancing in virtualized data centers. In: 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–7. IEEE (2013) Nine, M.S.Z., Azad, M.A.K., Abdullah, S., Rahman, R.M.: Fuzzy logic based dynamic load balancing in virtualized data centers. In: 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–7. IEEE (2013)
5.
Zurück zum Zitat Wang, T., Ma, H., Zhou, Y., Zhang, R., Song, Z.: Fully accountable data sharing for pay-as-you-go cloud scenes. IEEE Trans. Depend. Secur. Comput. 18, 2005–2016 (2019)CrossRef Wang, T., Ma, H., Zhou, Y., Zhang, R., Song, Z.: Fully accountable data sharing for pay-as-you-go cloud scenes. IEEE Trans. Depend. Secur. Comput. 18, 2005–2016 (2019)CrossRef
6.
Zurück zum Zitat Guo, J., Song, Z., Cui, Y., Liu, Z., Ji, Y.: Energy-efficient resource allocation for multi-user mobile edge computing. In: GLOBECOM—2017 IEEE Global Communications Conference, Singapore, pp. 1–7 (2017) Guo, J., Song, Z., Cui, Y., Liu, Z., Ji, Y.: Energy-efficient resource allocation for multi-user mobile edge computing. In: GLOBECOM—2017 IEEE Global Communications Conference, Singapore, pp. 1–7 (2017)
7.
Zurück zum Zitat Dutreilh, X., Kirgizov, S., Melekhova, O., Malenfant, J., Rivierre, N., Truck, I.: Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In: ICAS 2011, The Seventh International Conference on Autonomic and Autonomous Systems, Venice, pp. 67–74 (2011) Dutreilh, X., Kirgizov, S., Melekhova, O., Malenfant, J., Rivierre, N., Truck, I.: Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In: ICAS 2011, The Seventh International Conference on Autonomic and Autonomous Systems, Venice, pp. 67–74 (2011)
8.
Zurück zum Zitat Barrett, E., Howley, E., Duggan, J.: Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr. Comput.: Pract. Exp. 25(12), 1656–1674 (2013)CrossRef Barrett, E., Howley, E., Duggan, J.: Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr. Comput.: Pract. Exp. 25(12), 1656–1674 (2013)CrossRef
9.
Zurück zum Zitat Chen, Z., Hu, J., Min, G.: Learning-based resource allocation in cloud data center using advantage actor-critic. In: ICC—2019 IEEE International Conference on Communications, Singapore, pp. 1–6 (2019) Chen, Z., Hu, J., Min, G.: Learning-based resource allocation in cloud data center using advantage actor-critic. In: ICC—2019 IEEE International Conference on Communications, Singapore, pp. 1–6 (2019)
10.
Zurück zum Zitat Zhao, X., Wang, X., Ma, L., Zong, G.: Fuzzy approximation based asymptotic tracking control for a class of uncertain switched nonlinear systems. IEEE Trans. Fuzzy Syst. 28(4), 632–644 (2020)CrossRef Zhao, X., Wang, X., Ma, L., Zong, G.: Fuzzy approximation based asymptotic tracking control for a class of uncertain switched nonlinear systems. IEEE Trans. Fuzzy Syst. 28(4), 632–644 (2020)CrossRef
11.
Zurück zum Zitat Zhang, X., Biagioni, D., Cai, M., Graf, P., Rahman, S.: An edge-cloud integrated solution for buildings demand response using reinforcement learning. IEEE Trans. Smart Grid 12, 420–431 (2020)CrossRef Zhang, X., Biagioni, D., Cai, M., Graf, P., Rahman, S.: An edge-cloud integrated solution for buildings demand response using reinforcement learning. IEEE Trans. Smart Grid 12, 420–431 (2020)CrossRef
12.
Zurück zum Zitat Liu, C.H., Dai, Z., Zhao, Y., Crowcroft, J., Wu, D.O., Leung, K.: Distributed and energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning. IEEE Trans. Mob. Comput. 20, 130–146 (2019)CrossRef Liu, C.H., Dai, Z., Zhao, Y., Crowcroft, J., Wu, D.O., Leung, K.: Distributed and energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning. IEEE Trans. Mob. Comput. 20, 130–146 (2019)CrossRef
13.
Zurück zum Zitat Dane, L., Gurkan, D.: Netforager: Geographically-distributed network performance monitoring of web applications. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0142–0149. IEEE (2020) Dane, L., Gurkan, D.: Netforager: Geographically-distributed network performance monitoring of web applications. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0142–0149. IEEE (2020)
14.
Zurück zum Zitat Chen, L., Li, B., Li, B.: Allocating bandwidth in datacenter networks: a survey. J. Comput. Sci. Technol. 29(5), 910–917 (2014)CrossRef Chen, L., Li, B., Li, B.: Allocating bandwidth in datacenter networks: a survey. J. Comput. Sci. Technol. 29(5), 910–917 (2014)CrossRef
15.
Zurück zum Zitat Mattos, D.M., Ferraz, L.H.G., Costa, L.H.M., Duarte, O.C.M.: Evaluating virtual router performance for a pluralist future Internet. In: Proceedings of the 3rd International Conference on Information and Communication Systems, Irbid, Jordan, pp. 1–7 (2012) Mattos, D.M., Ferraz, L.H.G., Costa, L.H.M., Duarte, O.C.M.: Evaluating virtual router performance for a pluralist future Internet. In: Proceedings of the 3rd International Conference on Information and Communication Systems, Irbid, Jordan, pp. 1–7 (2012)
16.
Zurück zum Zitat Naseri, T.S., Gharehchopogh, F.S.: A feature selection based on the farmland fertility algorithm for improved intrusion detection systems. J. Netw. Syst. Manag. 30(3), 40 (2022)CrossRef Naseri, T.S., Gharehchopogh, F.S.: A feature selection based on the farmland fertility algorithm for improved intrusion detection systems. J. Netw. Syst. Manag. 30(3), 40 (2022)CrossRef
17.
Zurück zum Zitat Ali-Eldin, A., Tordsson, J., Elmroth, E.: An adaptive hybrid elasticity controller for cloud infrastructures. In: Proceedings of Network Operations and Management Symposium (NOMS), pp. 204–212 (2012) Ali-Eldin, A., Tordsson, J., Elmroth, E.: An adaptive hybrid elasticity controller for cloud infrastructures. In: Proceedings of Network Operations and Management Symposium (NOMS), pp. 204–212 (2012)
18.
Zurück zum Zitat Popa, L., Kumar, G., Chowdhury, M., Krishnamurthy, A., Ratnasamy, S., Stoica, I.: Faircloud: Sharing the network in cloud computing. In: Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 187–198 (2012) Popa, L., Kumar, G., Chowdhury, M., Krishnamurthy, A., Ratnasamy, S., Stoica, I.: Faircloud: Sharing the network in cloud computing. In: Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 187–198 (2012)
19.
Zurück zum Zitat Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. J. Grid Comput. 12(4), 559–592 (2014)CrossRef Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. J. Grid Comput. 12(4), 559–592 (2014)CrossRef
20.
Zurück zum Zitat Patikirikorala, T., Colman, A.: Feedback controllers in the cloud. In: Proceedings of Asia-Pacific Software Engineering Conference (APSEC). SN, pp. 1–6 (2010) Patikirikorala, T., Colman, A.: Feedback controllers in the cloud. In: Proceedings of Asia-Pacific Software Engineering Conference (APSEC). SN, pp. 1–6 (2010)
21.
Zurück zum Zitat Heinze, T., Pappalardo, V., Jerzak, Z., Fetzer, C.: Auto-scaling techniques for elastic data stream processing. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (DEBS), pp. 318–321. Association for Computing Machinery, New York, NY, USA (2014) Heinze, T., Pappalardo, V., Jerzak, Z., Fetzer, C.: Auto-scaling techniques for elastic data stream processing. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (DEBS), pp. 318–321. Association for Computing Machinery, New York, NY, USA (2014)
22.
Zurück zum Zitat Santos Filho, R.H., Ferreira, T.N., Mattos, D.M., Medeiros, D.S.: A lightweight reinforcement-learning-based mechanism for bandwidth provisioning on multitenant data center. In: 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 331–336. IEEE (2020) Santos Filho, R.H., Ferreira, T.N., Mattos, D.M., Medeiros, D.S.: A lightweight reinforcement-learning-based mechanism for bandwidth provisioning on multitenant data center. In: 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 331–336. IEEE (2020)
23.
Zurück zum Zitat Glorennec, P.Y., Jouffe, L.: Fuzzy q-learning. In: Proceedings of 6th International Fuzzy Systems Conference, vol. 2, pp. 659–662. IEEE (1997) Glorennec, P.Y., Jouffe, L.: Fuzzy q-learning. In: Proceedings of 6th International Fuzzy Systems Conference, vol. 2, pp. 659–662. IEEE (1997)
24.
Zurück zum Zitat Kiumarsi, B., Vamvoudakis, K.G., Modares, H., Lewis, F.L.: Optimal and autonomous control using reinforcement learning: A survey. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2042–2062 (2017)MathSciNetCrossRef Kiumarsi, B., Vamvoudakis, K.G., Modares, H., Lewis, F.L.: Optimal and autonomous control using reinforcement learning: A survey. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2042–2062 (2017)MathSciNetCrossRef
25.
Zurück zum Zitat Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proc. Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, pp. 2094–2100 (2016) Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proc. Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, pp. 2094–2100 (2016)
27.
Zurück zum Zitat Navin, N.K., Sharma, R.: A fuzzy reinforcement learning approach to thermal unit commitment problem. Neural Comput. Appl. 31(3), 737–750 (2019)CrossRef Navin, N.K., Sharma, R.: A fuzzy reinforcement learning approach to thermal unit commitment problem. Neural Comput. Appl. 31(3), 737–750 (2019)CrossRef
28.
Zurück zum Zitat Kofinas, P., Dounis, A., Vouros, G.: Fuzzy q-learning for multi-agent decentralized energy management in microgrids. Appl. Energy 219, 53–67 (2018)CrossRef Kofinas, P., Dounis, A., Vouros, G.: Fuzzy q-learning for multi-agent decentralized energy management in microgrids. Appl. Energy 219, 53–67 (2018)CrossRef
29.
Zurück zum Zitat Jamshidi, P., Sharifloo, A., Pahl, C., Arabnejad, H., Metzger, A., Estrada, G.: Fuzzy self-learning controllers for elasticity management in dynamic cloud architectures. In: 2016 12th International ACM SIGSOFT Conference on Quality of Software Architectures (QoSA), pp. 70–79. IEEE (2016) Jamshidi, P., Sharifloo, A., Pahl, C., Arabnejad, H., Metzger, A., Estrada, G.: Fuzzy self-learning controllers for elasticity management in dynamic cloud architectures. In: 2016 12th International ACM SIGSOFT Conference on Quality of Software Architectures (QoSA), pp. 70–79. IEEE (2016)
30.
Zurück zum Zitat Omidzade, F., Ghodousi, H., Shahverdi, K.: Comparing fuzzy SARSA learning and ant colony optimization algorithms in water delivery scheduling under water shortage conditions. J. Irrig. Drain. Eng. 146(9), 04020028 (2020)CrossRef Omidzade, F., Ghodousi, H., Shahverdi, K.: Comparing fuzzy SARSA learning and ant colony optimization algorithms in water delivery scheduling under water shortage conditions. J. Irrig. Drain. Eng. 146(9), 04020028 (2020)CrossRef
31.
Zurück zum Zitat Wibowo, F.X.A., Gregory, M.A.: Updating guaranteed bandwidth in multi-domain software defined networks. In: 2017 27th International Telecommunication Networks and Applications Conference (ITNAC), pp. 1–6 (2017) Wibowo, F.X.A., Gregory, M.A.: Updating guaranteed bandwidth in multi-domain software defined networks. In: 2017 27th International Telecommunication Networks and Applications Conference (ITNAC), pp. 1–6 (2017)
32.
Zurück zum Zitat Filho, R.H.S., Ferreira, T.N., Mattos, D.M.F., Medeiros, D.S.V.: A rapid fuzzy controller for decentralized bandwidth provisioning on a multitenant data center. In: Proceedings of Network of Future (NOF), pp. 1–8 (2020). To appear Filho, R.H.S., Ferreira, T.N., Mattos, D.M.F., Medeiros, D.S.V.: A rapid fuzzy controller for decentralized bandwidth provisioning on a multitenant data center. In: Proceedings of Network of Future (NOF), pp. 1–8 (2020). To appear
33.
Zurück zum Zitat Dane, L., Gurkan, D.: Netforager: Geographically-distributed network performance monitoring of web applications. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0142–0149 (2020) Dane, L., Gurkan, D.: Netforager: Geographically-distributed network performance monitoring of web applications. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0142–0149 (2020)
34.
Zurück zum Zitat Wette, P., Dräxler, M., Schwabe, A., Wallaschek, F., Zahraee, M.H., Karl, H.: Maxinet: Distributed emulation of software-defined networks. In: 2014 IFIP Networking Conference, pp. 1–9 (2014) Wette, P., Dräxler, M., Schwabe, A., Wallaschek, F., Zahraee, M.H., Karl, H.: Maxinet: Distributed emulation of software-defined networks. In: 2014 IFIP Networking Conference, pp. 1–9 (2014)
35.
Zurück zum Zitat Lantz, B., Heller, B., McKeown, N.: A network in a laptop: Rapid prototyping for software-defined networks. In: Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks. Hotnets-IX, pp. 19–1196, New York, NY, USA (2010) Lantz, B., Heller, B., McKeown, N.: A network in a laptop: Rapid prototyping for software-defined networks. In: Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks. Hotnets-IX, pp. 19–1196, New York, NY, USA (2010)
36.
Zurück zum Zitat Turner, A.: Tcpreplay. http://tcpreplay. synfin. net/trac/ (2011) Turner, A.: Tcpreplay. http://​tcpreplay.​ synfin. net/trac/ (2011)
37.
Zurück zum Zitat Networking, C.V.: Cisco global cloud index: Forecast and methodology, 2015-2020. white paper. Cisco Public, San Jose (2016) Networking, C.V.: Cisco global cloud index: Forecast and methodology, 2015-2020. white paper. Cisco Public, San Jose (2016)
38.
Zurück zum Zitat Wu, C., Yoshinaga, T., Chen, X., Zhang, L., Ji, Y.: Cluster-based content distribution integrating ITE and IEEE 802.11 p with fuzzy logic and Q-learning. IEEE Comput. Intell. Mag. 13(1), 41–50 (2018)CrossRef Wu, C., Yoshinaga, T., Chen, X., Zhang, L., Ji, Y.: Cluster-based content distribution integrating ITE and IEEE 802.11 p with fuzzy logic and Q-learning. IEEE Comput. Intell. Mag. 13(1), 41–50 (2018)CrossRef
39.
Zurück zum Zitat Bai, W., Chen, L., Chen, K., Han, D., Tian, C., Wang, H.: Information-agnostic flow scheduling for commodity data centers. In: 12th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 15), pp. 455–468 (2015) Bai, W., Chen, L., Chen, K., Han, D., Tian, C., Wang, H.: Information-agnostic flow scheduling for commodity data centers. In: 12th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 15), pp. 455–468 (2015)
40.
Zurück zum Zitat Chen, L., Lingys, J., Chen, K., Liu, F.: Auto: Scaling deep reinforcement learning for datacenter-scale automatic traffic optimization. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp. 191–205 (2018) Chen, L., Lingys, J., Chen, K., Liu, F.: Auto: Scaling deep reinforcement learning for datacenter-scale automatic traffic optimization. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp. 191–205 (2018)
Metadaten
Titel
An Efficient and Decentralized Fuzzy Reinforcement Learning Bandwidth Controller for Multitenant Data Centers
verfasst von
Reiner H. Santos Filho
Tadeu N. Ferreira
Diogo M. F. Mattos
Dianne S. V. Medeiros
Publikationsdatum
01.10.2022
Verlag
Springer US
Erschienen in
Journal of Network and Systems Management / Ausgabe 4/2022
Print ISSN: 1064-7570
Elektronische ISSN: 1573-7705
DOI
https://doi.org/10.1007/s10922-022-09667-3

Weitere Artikel der Ausgabe 4/2022

Journal of Network and Systems Management 4/2022 Zur Ausgabe

Premium Partner