Top

Journal of Network and Systems Management

Published in:

01-01-2022

Centrality Based Congestion Detection Using Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN

Authors: Deva Priya Isravel, Salaja Silas, Elijah Blessing Rajsingh

Published in: Journal of Network and Systems Management | Issue 1/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The rising number of users and the demand for more diverse and specialized applications have led to a tremendous increase in network traffic. Managing diverse traffic demands from numerous applications is a challenging task for the existing traditional networking architecture. Hybrid software defined network is widely used to simplify operations providing flexible traffic management and automation. However, managing dynamic traffic demands and routing traffic flows is a challenging task. Therefore, in this paper, a centrality based Q-learning routing traffic engineering method for congestion detection and optimized traffic routing is proposed. The proposed method uses the reinforcement Q-learning algorithm to find an optimal path for routing the traffic. The centrality measures of the nodes are computed and ranked using the simple additive weighted method to detect the top k high-risk nodes. Simulations are carried out under different network scenarios for various traffic profiles. The results show that the proposed method outperformed the existing methods in terms of path length, delay, link utilization, throughput and computation time.

previous article A Scalable Hierarchically Distributed Architecture for Next-Generation Applications

next article Model-Based Performability and Dependability Evaluation of a System with VM Migration as Rejuvenation in the Presence of Bursty Workloads

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Ahvar, E., Ahvar, S., Raza, S.M., Vilchez, J.M.S., Lee, G.M.: Next generation of SDN in cloud-fog for 5G and beyond-enabled applications: opportunities and challenges. Network 1, 28–49 (2021). https://doi.org/10.3390/network1010004CrossRef

Singh, S., Jha, R.K.: A survey on software defined networking: architecture for next generation network. J. Netw. Syst. Manag. 25, 321–374 (2017). https://doi.org/10.1007/s10922-016-9393-9CrossRef

Vissicchio, S., Vanbever, L., Bonaventure, O.: Opportunities and research challenges of hybrid software defined networks. ACM SIGCOMM Comput. Commun. Rev. 44, 70–75 (2014). https://doi.org/10.1145/2602204.2602216CrossRef

Kreutz, D., Ramos, F.M., Veríssimo, P.E., Rothenberg, C.E., Azodolmolky, S., Uhlig, S.: Software-defined networking : a comprehensive survey. Proc. IEEE. 103, 14–76 (2015). https://doi.org/10.1109/JPROC.2014.2371999CrossRef

Ahmad, S., Mir, A.H.: Scalability, consistency, reliability and security in SDN controllers: a survey of diverse SDN controllers. J. Netw. Syst. Manag. (2021). https://doi.org/10.1007/s10922-020-09575-4CrossRef

Sanvito, D., Filippini, I., Capone, A., Paris, S., Leguay, J.: Clustered robust routing for traffic engineering in software-defined networks. Comput. Commun. 144, 175–187 (2019). https://doi.org/10.1016/j.comcom.2019.06.002CrossRef

Chen, F., Wu, C., Hong, X., Lu, Z., Wang, Z., Lin, C.: Engineering traffic uncertainty in the OpenFlow data plane. Proc. IEEE INFOCOM. (2016). https://doi.org/10.1109/INFOCOM.2016.752461CrossRef

Takahashi, Y., Ishibashi, K., Tsujino, M., Kamiyama, N., Shiomoto. K., Otoshi, T., Ohsita, Y., Murata, M.: Separating predictable and unpredictable flows via dynamic flow mining for effective traffic engineering. IEEE Int. Conf. Commun. (2016). https://doi.org/10.1109/ICC.2016.7511371

Özçevik, M.E., Canberk, B., Duong, T.Q.: End to end delay modeling of heterogeneous traffic flows in software defined 5G networks. Ad Hoc Netw. 60, 26–39 (2017). https://doi.org/10.1016/j.adhoc.2017.02.006CrossRef

10.

Kim, H., Feamster, N.: Improving network management with software defined networking. IEEE Commun. Mag. 51, 114–119 (2013). https://doi.org/10.1109/MCOM.2013.6461195CrossRef

11.

Amin, R., Reisslein, M., Shah, N.: Hybrid SDN networks: a survey of existing approaches. IEEE Commun. Surv. Tutor. 20(4), 3259–3306 (2018). https://doi.org/10.1109/COMST.2018.2837161CrossRef

12.

Vissicchio, S., Vanbever, L., Cittadini, L., Xie, G., Bonaventure, O.: Safe updates of hybrid SDN networks. IEEE/ACM Trans. Netw. 25(3), 1649–1662 (2017). https://doi.org/10.1109/TNET.2016.2642586CrossRef

13.

Casas-Velasco, D.M., Rendon, O.M.C., da Fonseca, N.L.S.: Intelligent routing based on reinforcement learning for software-defined networking. IEEE Trans. Netw. Manag. 18(1), 870–881 (2021). https://doi.org/10.1109/TNSM.2020.3036911CrossRef

14.

Wang, J., Yang, Y., Xiao, L., Nahrstedt, K.: Edge-based traffic engineering for OSPF networks. Comput. Networks. 48(4), 605–625 (2005). https://doi.org/10.1016/j.comnet.2004.11.008CrossRefMATH

15.

Chiesa, M., Kindler, G., Schapira, M.: Traffic engineering with equal-cost-multipath: an algorithmic perspective. IEEE/ACM Trans. Netw. 25(2), 779–792 (2017). https://doi.org/10.1109/TNET.2016.2614247CrossRef

16.

Guo, Y., Wang, Z., Yin, X., Shi, X., Wu, J.: Traffic Engineering in SDN/OSPF Hybrid Network. In: Proc. - IEEE 22nd Int. Conf. on Netw. Protoc. (2014) https://doi.org/10.1109/ICNP.2014.90

17.

Buriol, L.S., Resende, M.G.C., Ribeiro, C.C., Thorup, M.: A hybrid genetic algorithm for the weight setting problem in OSPF/IS-IS routing. Netw. 46(1), 36–56 (2005). https://doi.org/10.1002/net.20070MathSciNetCrossRefMATH

18.

Fortz, B., Thorup, M.: Internet traffic engineering by optimizing OSPF weights. Internet traffic engineering by optimizing OSPF weights. Proc. IEEE INFOCOM. 2, 519–528 (2000). https://doi.org/10.1109/INFCOM.2000.832225.

19.

Isravel, D.P., Silas, S., Rajsingh, E.B.: Preferential multi-attribute rank based forwarding node selection in software defined networks. In: Proc.- 3rd Int. Conf. Comput. Commun. Technol., pp. 73–79 (2019). https://doi.org/10.1109/ICCCT2.2019.8824798

20.

Mishra, A., Sahoo, A., Dalvi, B., Zhu, T.: WOSPF: a traffic engineering solution for OSPF networks. In: Proc.- IEEE Glob. Commun. Conf. GLOBECOM (2016). https://doi.org/10.1109/GLOCOM.2016.7842393

21.

Shirmarz, A., Ghaffari, A.: An adaptive greedy flow routing algorithm for performance improvement in software—defined network. Int. J. Numer. Model. (2019). https://doi.org/10.1002/jnm.2676CrossRef

22.

Majidi, A., Jahanbakhsh, N., Gao, X., Zheng, J., Chen, G.: DC-ECN: a machine-learning based dynamic threshold control scheme for ECN marking in DCN. Comput. Commun. (2019). https://doi.org/10.1016/j.comcom.2019.10.028CrossRef

23.

Wang, H., Xu, H., Huang, L., Wang, J., Yang, X.: Load-balancing routing in software defined networks with multiple controllers. Comput. Netw. 141, 82–91 (2018). https://doi.org/10.1016/j.comnet.2018.05.012CrossRef

24.

Nasimi, M., Habibi, M.A., Han, B., Schotten, H.D.: Edge-assisted congestion control mechanism for 5G network using software-defined networking. In: Proc. -Int. Symp. Wirel. Commun. Syst. 1–5, (2018). https://doi.org/10.1109/ISWCS.2018.8491233

25.

Handigol, N., Seetharaman, S., McKeown, N., Johari, R.: Plug-n-Serve: load-balancing web traffic using OpenFlow. ACM Sigcomm Demo (2009).

26.

Chu, C.Y., Xi, K., Luo, M., Chao, H.J.: Congestion-aware single link failure recovery in hybrid SDN networks. Proc. IEEE INFOCOM. 26, 1086–1094 (2015). https://doi.org/10.1109/INFOCOM.2015.7218482CrossRef

27.

Zakia, U., Yedder, H.B.: Dynamic load balancing in SDN-based data center networks. In: IEEE Annu. Inf. Techno., Electron. and Mob. Commun. Conf. (IEMCON), pp. 242–247 (2017). https://doi.org/10.1109/IEMCON.2017.8117206

28.

Tajiki, M.M., Akbari, B., Shojafar, M., Mokari, N.: Joint QoS and congestion control based on traffic prediction in SDN. Appl. Sci. 7(12), 1–15 (2017). https://doi.org/10.3390/app7121265CrossRef

29.

Le, D.C., Zincir-Heywood, N.: A frontier: dependable, reliable and secure machine learning for network/system management. J. Netw. Syst. Manag. 28, 827–849 (2020). https://doi.org/10.1007/s10922-020-09512-5CrossRef

30.

Xie, J., Yu, F.R., Huang, T., Xie, R., Liu, J., Wang, C., Liu, Y.: A survey of machine learning techniques applied to software defined networking (SDN): research issues and challenges. IEEE Commun. Surv. Tutorials. 21(1), 393–430 (2019). https://doi.org/10.1109/COMST.2018.2866942CrossRef

31.

Dong, M., Li, Q., Zarchy, D., Godfrey, P.B., Schapira, M.: PCC: Re-architecting congestion control for consistent high performance. In: Proc. 12th USENIX Symp. Networked Syst. Des. Implement. NSDI '15, pp. 395–408 (2015).

32.

Dong, M., Tong, M., Zarchy, D., Arslan, E., Gilad, Y., Godfrey, B., Schapira, M.: PCC Vivace : online-learning congestion control. In: Proc. 15th USENIX Symp. Networked Syst. Des. Implement. (NSDI '18), pp. 343–356 (2018).

33.

Nguyen, T.T.T., Armitage, G.: A survey of techniques for internet traffic classification using machine learning. IEEE Commun. Surv. Tutorials. 10(4), 56–76 (2008). https://doi.org/10.1109/SURV.2008.080406CrossRef

34.

Azzouni, A., Boutaba, R., Pujolle, G.: NeuRoute : Predictive dynamic routing for software-defined networks. In: Proc. - 13th Int. Conf. Netw. Serv. Manag. (CNSM), pp. 1–6 (2017). https://doi.org/10.23919/CNSM.2017.8256059

35.

Assefa, B.G., Ozkasap, O.: MER-SDN : machine learning framework for traffic aware energy efficient routing in SDN. In: IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech), pp. 74–980 (2018). https://doi.org/10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.000-1

36.

Choi, S.P.M., Yeung, D.-Y.: Predictive Q-routing: a memory-based reinforcement learning approach to adaptive traffic control. Adv. Neural Inf. Process. Syst. 8, 945–951 (1996)

37.

Chavula, J., Densmore, M., Suleman, H.: Using SDN and reinforcement learning for traffic engineering in ubuntunet alliance. In: Proc. Int. Conf. Adv. Comput. Commun. Eng. (ICACCE), pp. 349–355 (2016). https://doi.org/10.1109/ICACCE.2016.8073774

38.

Kong, Y., Zang, H., Ma, X.: Improving TCP congestion control with machine intelligence. In: Proc. Work. Network Meets AI and ML, pp. 60–66 (2018). https://doi.org/10.1145/3229543.3229550

39.

Lin, S., Akyildiz, I. F., Wang, P., Luo, M.: QoS-aware adaptive routing in multi-layer hierarchical software defined networks : a reinforcement learning approach. In: IEEE Int. Conf. Serv. Comput., pp. 25–33 (2016). https://doi.org/10.1109/SCC.2016.12

40.

Yu, C., Lan, J., Guo, Z., Hu, Y.: DROM: optimizing the routing in software-defined networks with deep reinforcement learning. IEEE Access 6, 64533–64539 (2018). https://doi.org/10.1109/ACCESS.2018.2877686CrossRef

41.

Tessler, C., Shpigelman, Y., Dalal, G., Mandelbaum, A., Kazakov, D.H., Fuhrer, B., Chechik, G., Mannor, S.: Reinforcement learning for datacenter congestion control (2021). arXiv:2102.09337

42.

Bachl, M., Zseby, T., Fabini, J.: Rax: deep reinforcement learning for congestion control. IEEE Int. Conf. Commun. (2019). https://doi.org/10.1109/ICC.2019.8761187CrossRef

43.

Lei, K., Liang, Y., Li, W.: Congestion control in SDN-based networks via multi-task deep reinforcement learning. IEEE Netw. 34, 28–34 (2020). https://doi.org/10.1109/MNET.011.1900408CrossRef

44.

Zhang, J., Ye, M., Guo, Z., Yen, C.-Y., Choa, H.J.: CFR-RL: Traffic engineering with reinforcement learning in SDN. IEEE J. Sel. Areas Commun. 38(10), 2249–2259 (2020). https://doi.org/10.1109/JSAC.2020.3000371CrossRef

45.

Agryzkov, T., Oliver, J.L., Tortosa, L., Vicent, J.: A new betweenness centrality measure based on an algorithm for ranking the nodes of a network. Appl. Math. Comput. 244, 467–478 (2014). https://doi.org/10.1016/j.amc.2014.07.026MathSciNetCrossRefMATH

46.

Saxena, A., Gera, R., Iyengar, S.R.S.: A faster method to estimate closeness centrality ranking (2017). arXiv:1706.02083

47.

Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. J. Am. Society Inf. Sci. Technol. 58(7), 1019–1031 (2007)CrossRef

48.

Moradi, P., Shiri, M.E., Entezari, N.: Automatic skill acquisition in reinforcement learning agents using agents using connection bridge centrality. In: Kim, T., Vasilakos, T., Sakurai, K., Xiao, Y., Zhao, G., Ślęzak, D. (eds) Communication and Networking. FGCN 2010. Commun. Comput. Inf. Sci. 120, pp. 51–62. Springer, Berlin (2010). https://doi.org/10.1007/978-3-642-17604-3

49.

Zhang, J., Luo, Y.: Degree centralityIn, betweenness centrality , and closeness centrality in social network. In: Proc. - 2nd Int. Conf. Model. Simul. Appl. Math., vol. 132, pp. 300–303 (2017). https://doi.org/10.2991/msam-17.2017.68

50.

Okamoto, K., Chen, W., Li, X.Y.: Ranking of closeness centrality for large-scale social networks. In: Preparata, F.P., Wu, X., Yin, J. (eds) Frontiers in Algorithmics. FAW 2008. Lecture Notes in Comput. Sci. 5059, pp. 186–195. Springer, Berlin. https://doi.org/10.1007/978-3-540-69311-6_21

51.

Benzi, M., Klymko, C.: On the limiting behavior of parameter-dependent network centrality measures. SIAM J. Matrix Anal. Appl. 36(2), 686–706 (2015). https://doi.org/10.1137/130950550MathSciNetCrossRefMATH

52.

Kaliszewski, I., Podkopaev, D.: Simple additive weighting—a metamodel for multiple criteria decision analysis methods. Expert Syst. Appl. 54, 155–161 (2016). https://doi.org/10.1016/j.eswa.2016.01.042CrossRef

53.

Bhattacharyya, R., Bura, A., Rengarajan, D., Rumuly, M., Shakkottai, S., Kalathil, D., Mok, R.K.P., Dhamdhere, A.: QFlow : A reinforcement learning approach to high QoE video streaming over wireless networks. In: Proc. - 20th ACM Int. Symp. Mobile Ad Hoc Netw. Comput., pp. 251–260 (2019). https://doi.org/10.1145/3323679.3326523

Title: Centrality Based Congestion Detection Using Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN
Authors: Deva Priya Isravel
Salaja Silas
Elijah Blessing Rajsingh
Publication date: 01-01-2022
Publisher: Springer US
Published in: Journal of Network and Systems Management / Issue 1/2022
Print ISSN: 1064-7570
Electronic ISSN: 1573-7705
DOI: https://doi.org/10.1007/s10922-021-09627-3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2022

Deep Learning in IoT Intrusion Detection

Comparison of Machine Learning Techniques for VNF Resource Requirements Prediction in NFV

Intelligent Wireless Networks: Challenges and Future Research Topics

Model-Based Performability and Dependability Evaluation of a System with VM Migration as Rejuvenation in the Presence of Bursty Workloads

PassMon: A Technique for Password Generation and Strength Estimation

Towards Model Generalization for Intrusion Detection: Unsupervised Machine Learning Techniques

Premium Partner