Skip to main content
Erschienen in: Knowledge and Information Systems 9/2021

21.07.2021 | Regular Paper

Deep reinforcement learning-based resource allocation and seamless handover in multi-access edge computing based on SDN

verfasst von: Chunlin Li, Yong Zhang, Youlong Luo

Erschienen in: Knowledge and Information Systems | Ausgabe 9/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

With the access devices that are densely deployed in multi-access edge computing environments, users frequently switch access devices when moving, which causes the imbalance of network load and the decline of service quality. To solve the problems above, a seamless handover scheme for wireless access points based on perception is proposed. First, a seamless handover model based on load perception is proposed to solve the unbalanced network load, in which a seamless handover algorithm for wireless access points is used to calculate the access point with the highest weight, and a software-defined network controller controls the switching process. A joint allocation method of communication and computing resources based on deep reinforcement learning is proposed to minimize the terminal energy consumption and the system delay. A resource allocation model is based on minimizing terminal energy consumption, and system delay is built. The optimal value of task offloading decision and resource allocation vector are calculated with deep reinforcement learning. Experimental results show that the proposed method can reduce the network load and the task execution cost.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Moazenzadeh R, Mohammadi B (2019) Assessment of bio-inspired metaheuristic optimisation algorithms for estimating soil temperature. Geoderma 353:152–171CrossRef Moazenzadeh R, Mohammadi B (2019) Assessment of bio-inspired metaheuristic optimisation algorithms for estimating soil temperature. Geoderma 353:152–171CrossRef
2.
Zurück zum Zitat Donepudi S, Garg S, Agarwal K, et al. (2014) Dynamic multi-access wireless network virtualization Donepudi S, Garg S, Agarwal K, et al. (2014) Dynamic multi-access wireless network virtualization
3.
Zurück zum Zitat Kim H, Feamster N (2013) Improving network management with software defined networking. IEEE Commun Mag 51(2):115–169CrossRef Kim H, Feamster N (2013) Improving network management with software defined networking. IEEE Commun Mag 51(2):115–169CrossRef
4.
Zurück zum Zitat Baktir AC, Ozgovde A, Ersoy C (2017) How can edge computing benefit from software-defined networking: a survey, use cases, and future directions. IEEE Commun Surv Tutor 19(4):2359–2391CrossRef Baktir AC, Ozgovde A, Ersoy C (2017) How can edge computing benefit from software-defined networking: a survey, use cases, and future directions. IEEE Commun Surv Tutor 19(4):2359–2391CrossRef
5.
Zurück zum Zitat Li C, Song M, Zhang M, Luo Y (2020) Effective replica management for improving reliability and availability in edge-cloud computing environment. J Parallel Distrib Comput 143:107–128CrossRef Li C, Song M, Zhang M, Luo Y (2020) Effective replica management for improving reliability and availability in edge-cloud computing environment. J Parallel Distrib Comput 143:107–128CrossRef
6.
Zurück zum Zitat Li C, Song M, Yu C, Luo YL (2021) Mobility and marginal gain based content caching and placement for cooperative edge-cloud computing. Inf Sci 548(16):153–176CrossRef Li C, Song M, Yu C, Luo YL (2021) Mobility and marginal gain based content caching and placement for cooperative edge-cloud computing. Inf Sci 548(16):153–176CrossRef
7.
Zurück zum Zitat Zhang K, Mao Y, Leng S et al (2017) Mobile-edge computing for vehicular networks: a promising network paradigm with predictive off-loading. IEEE Veh Technol Mag 12(2):36–44CrossRef Zhang K, Mao Y, Leng S et al (2017) Mobile-edge computing for vehicular networks: a promising network paradigm with predictive off-loading. IEEE Veh Technol Mag 12(2):36–44CrossRef
8.
Zurück zum Zitat Huang A, Nikaein N, Stenbock T, et al. (2017) Low latency MEC framework for SDN-based LTE/LTE-A networks. In: 2017 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6 Huang A, Nikaein N, Stenbock T, et al. (2017) Low latency MEC framework for SDN-based LTE/LTE-A networks. In: 2017 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6
9.
Zurück zum Zitat Katov AN, Mihovska A, Prasad NR (2015) Hybrid SDN architecture for resource consolidation in MPLS networks. In: Wireless telecommunications symposium. IEEE Katov AN, Mihovska A, Prasad NR (2015) Hybrid SDN architecture for resource consolidation in MPLS networks. In: Wireless telecommunications symposium. IEEE
10.
Zurück zum Zitat Schiller E, Nikaein N, Kalogeiton E et al (2018) CDS-MEC: NFV/SDN-based application management for MEC in 5G systems. Comput Netw 135:96–107CrossRef Schiller E, Nikaein N, Kalogeiton E et al (2018) CDS-MEC: NFV/SDN-based application management for MEC in 5G systems. Comput Netw 135:96–107CrossRef
11.
Zurück zum Zitat Peng H, Ye Q, Shen XS (2019) SDN-based resource management for autonomous vehicular networks: a multi-access edge computing approach. IEEE Wirel Commun 26(4):156–162CrossRef Peng H, Ye Q, Shen XS (2019) SDN-based resource management for autonomous vehicular networks: a multi-access edge computing approach. IEEE Wirel Commun 26(4):156–162CrossRef
12.
Zurück zum Zitat Miladinovic I, Schefer-Wenzl S, Hirner H (2019) IoT architecture for smart cities leveraging machine learning and SDN. In: 2019 27th Telecommunications Forum. Washington: IEEE Computer Society Press, pp 1–4 Miladinovic I, Schefer-Wenzl S, Hirner H (2019) IoT architecture for smart cities leveraging machine learning and SDN. In: 2019 27th Telecommunications Forum. Washington: IEEE Computer Society Press, pp 1–4
13.
Zurück zum Zitat Xia W, Zhang J, Quek TQS et al (2020) Mobile edge cloud-based industrial internet of things: improving edge intelligence with hierarchical SDN controllers. IEEE Veh Technol Mag 15(1):36–45CrossRef Xia W, Zhang J, Quek TQS et al (2020) Mobile edge cloud-based industrial internet of things: improving edge intelligence with hierarchical SDN controllers. IEEE Veh Technol Mag 15(1):36–45CrossRef
14.
Zurück zum Zitat Wang J, Hu J, Min G et al (2019) Computation offloading in multi-access edge computing using a deep sequential model based on reinforcement learning. IEEE Commun Mag 57(5):64–69CrossRef Wang J, Hu J, Min G et al (2019) Computation offloading in multi-access edge computing using a deep sequential model based on reinforcement learning. IEEE Commun Mag 57(5):64–69CrossRef
15.
Zurück zum Zitat Tahaei H, Ko K, Seo W et al (2017) A QoE based trustable SDN framework for IoT devices in mobile edge computing. Springer, Singapore Tahaei H, Ko K, Seo W et al (2017) A QoE based trustable SDN framework for IoT devices in mobile edge computing. Springer, Singapore
16.
Zurück zum Zitat Yazdinejad A, Parizi RM, Dehghantanha A et al (2020) An energy-efficient SDN controller architecture for IoT networks with blockchain-based security. IEEE Trans Serv Comput 13:625–638CrossRef Yazdinejad A, Parizi RM, Dehghantanha A et al (2020) An energy-efficient SDN controller architecture for IoT networks with blockchain-based security. IEEE Trans Serv Comput 13:625–638CrossRef
17.
Zurück zum Zitat Bao W, Yuan D, Yang Z et al (2017) Follow me fog: toward seamless handover timing schemes in a fog computing environment. IEEE Commun Mag 55(11):72–78CrossRef Bao W, Yuan D, Yang Z et al (2017) Follow me fog: toward seamless handover timing schemes in a fog computing environment. IEEE Commun Mag 55(11):72–78CrossRef
18.
Zurück zum Zitat Yunoki K, Shinbo H (2018) Carry-on state service handover between edge hosts for latency strict applications in mobile networks. In: 2018 21st international symposium on wireless personal multimedia communications. Washington: IEEE Computer Society Press, pp 472–477 Yunoki K, Shinbo H (2018) Carry-on state service handover between edge hosts for latency strict applications in mobile networks. In: 2018 21st international symposium on wireless personal multimedia communications. Washington: IEEE Computer Society Press, pp 472–477
19.
Zurück zum Zitat Neto AJV, Silva FSD, Neto EDP et al (2020) A taxonomy of DDoS attack mitigation approaches featured by SDN technologies in IoT scenarios. Sensors 20(11):3078CrossRef Neto AJV, Silva FSD, Neto EDP et al (2020) A taxonomy of DDoS attack mitigation approaches featured by SDN technologies in IoT scenarios. Sensors 20(11):3078CrossRef
20.
Zurück zum Zitat Bi Y, Han G, Lin C et al (2018) Mobility support for fog computing: an SDN approach. IEEE Commun Mag 56(5):53–59CrossRef Bi Y, Han G, Lin C et al (2018) Mobility support for fog computing: an SDN approach. IEEE Commun Mag 56(5):53–59CrossRef
21.
Zurück zum Zitat Zeljković E, Slamnik-Kriještorac N, Latré S et al (2019) ABRAHAM: machine learning backed proactive handover algorithm using SDN. IEEE Trans Netw Serv Manage 16(4):1522–1536CrossRef Zeljković E, Slamnik-Kriještorac N, Latré S et al (2019) ABRAHAM: machine learning backed proactive handover algorithm using SDN. IEEE Trans Netw Serv Manage 16(4):1522–1536CrossRef
22.
Zurück zum Zitat Yin X, Wang L (2017) A fast handover scheme for SDN based vehicular network. In: International conference on mobile ad-hoc and sensor networks. Berlin: Springer, pp 293–302 Yin X, Wang L (2017) A fast handover scheme for SDN based vehicular network. In: International conference on mobile ad-hoc and sensor networks. Berlin: Springer, pp 293–302
23.
Zurück zum Zitat Mouawad N, Naja R, Tohme S (2019) Fast and seamless handover in software defined vehicular networks. In: 2019 eleventh international conference on ubiquitous and future networks. Washington: IEEE Computer Society Press, pp 484–489 Mouawad N, Naja R, Tohme S (2019) Fast and seamless handover in software defined vehicular networks. In: 2019 eleventh international conference on ubiquitous and future networks. Washington: IEEE Computer Society Press, pp 484–489
24.
Zurück zum Zitat Zhang Y, Deng RH, Bertino E, et al. (2019) Robust and universal seamless handover authentication in 5G HetNets. In: IEEE transactions on dependable and secure computing, pp (99): 1–1 Zhang Y, Deng RH, Bertino E, et al. (2019) Robust and universal seamless handover authentication in 5G HetNets. In: IEEE transactions on dependable and secure computing, pp (99): 1–1
25.
Zurück zum Zitat Mohseni H, Eslamnour B (2019) Handover management for delay-sensitive IoT services on wireless software-defined network platforms. In: 2019 3rd international conference on internet of things and applications. Washington: IEEE Computer Society Press, pp 1–6 Mohseni H, Eslamnour B (2019) Handover management for delay-sensitive IoT services on wireless software-defined network platforms. In: 2019 3rd international conference on internet of things and applications. Washington: IEEE Computer Society Press, pp 1–6
26.
Zurück zum Zitat Bi Y, Han G, Lin C et al (2019) Mobility management for intro/inter domain handover in software-defined networks. IEEE J Sel Areas Commun 37(8):1739–1754CrossRef Bi Y, Han G, Lin C et al (2019) Mobility management for intro/inter domain handover in software-defined networks. IEEE J Sel Areas Commun 37(8):1739–1754CrossRef
27.
Zurück zum Zitat Zhong X, Wang X, Li L, et al. (2020) A cooperative learning framework for resource management in MEC: an ADMM perspective Zhong X, Wang X, Li L, et al. (2020) A cooperative learning framework for resource management in MEC: an ADMM perspective
28.
Zurück zum Zitat Lyu X, Tian H, Sengul C et al (2017) Multiuser joint task offloading and resource optimization in proximate clouds. IEEE Trans Veh Technol 66(4):1–1CrossRef Lyu X, Tian H, Sengul C et al (2017) Multiuser joint task offloading and resource optimization in proximate clouds. IEEE Trans Veh Technol 66(4):1–1CrossRef
29.
Zurück zum Zitat Tran TX, Pompili D (2018) Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Trans Veh Technol 68(1):856–868CrossRef Tran TX, Pompili D (2018) Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Trans Veh Technol 68(1):856–868CrossRef
30.
Zurück zum Zitat Cheng K, Teng Y, Sun W, et al. (2018) Energy-efficient joint offloading and wireless resource allocation strategy in multi-MEC server systems. In: 2018 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6 Cheng K, Teng Y, Sun W, et al. (2018) Energy-efficient joint offloading and wireless resource allocation strategy in multi-MEC server systems. In: 2018 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6
31.
Zurück zum Zitat Liang C, He Y, Yu FR, et al. (2017) Energy-efficient resource allocation in software-defined mobile networks with mobile edge computing and caching. In: 2017 IEEE conference on computer communications workshops. Washington: IEEE Computer Society Press, pp 121–126 Liang C, He Y, Yu FR, et al. (2017) Energy-efficient resource allocation in software-defined mobile networks with mobile edge computing and caching. In: 2017 IEEE conference on computer communications workshops. Washington: IEEE Computer Society Press, pp 121–126
32.
Zurück zum Zitat Wang P, Yao C, Zheng Z et al (2018) Joint task assignment, transmission, and computing resource allocation in multilayer mobile edge computing systems. IEEE Internet Things J 6(2):2872–2884CrossRef Wang P, Yao C, Zheng Z et al (2018) Joint task assignment, transmission, and computing resource allocation in multilayer mobile edge computing systems. IEEE Internet Things J 6(2):2872–2884CrossRef
33.
Zurück zum Zitat Qian LP, Shi B, Wu Y et al (2020) NOMA enabled mobile edge computing for internet of things via joint communication and computation resource allocations. IEEE Internet Things J 7(1):718–733CrossRef Qian LP, Shi B, Wu Y et al (2020) NOMA enabled mobile edge computing for internet of things via joint communication and computation resource allocations. IEEE Internet Things J 7(1):718–733CrossRef
34.
Zurück zum Zitat Yang Z, Liu Y, Chen Y, et al. (2019) Deep reinforcement learning in cache-aided MEC networks. In: ICC 2019–2019 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6 Yang Z, Liu Y, Chen Y, et al. (2019) Deep reinforcement learning in cache-aided MEC networks. In: ICC 2019–2019 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6
35.
Zurück zum Zitat Li C, Zhang Y, Zhiqiang H et al (2020) An effective scheduling strategy based on hypergraph partition in geographically distributed datacenters. Comput Networks 170:107096CrossRef Li C, Zhang Y, Zhiqiang H et al (2020) An effective scheduling strategy based on hypergraph partition in geographically distributed datacenters. Comput Networks 170:107096CrossRef
36.
Zurück zum Zitat Li C, Bai J, Yi C et al (2020) Resource and replica management strategy for optimizing financial cost and user experience in edge cloud computing system. Inf Sci 516:33–55MathSciNetCrossRef Li C, Bai J, Yi C et al (2020) Resource and replica management strategy for optimizing financial cost and user experience in edge cloud computing system. Inf Sci 516:33–55MathSciNetCrossRef
37.
Zurück zum Zitat Li C, Tang J, Ma T, Yang X, Luo Y (2020) A workflow job scheduling algorithm based on load balancing in distributed cloud. J Network Comput Appl 152:1518CrossRef Li C, Tang J, Ma T, Yang X, Luo Y (2020) A workflow job scheduling algorithm based on load balancing in distributed cloud. J Network Comput Appl 152:1518CrossRef
38.
Zurück zum Zitat Wang S, Xu J, Zhang N et al (2018) A survey on service migration in mobile edge computing. IEEE Access 6:23511–23528CrossRef Wang S, Xu J, Zhang N et al (2018) A survey on service migration in mobile edge computing. IEEE Access 6:23511–23528CrossRef
39.
Zurück zum Zitat Han Z, Lei T, Lu Z et al (2019) Artificial intelligence-based handover management for dense WLANs: a deep reinforcement learning approach. IEEE Access 7:31688–31701CrossRef Han Z, Lei T, Lu Z et al (2019) Artificial intelligence-based handover management for dense WLANs: a deep reinforcement learning approach. IEEE Access 7:31688–31701CrossRef
40.
Zurück zum Zitat Banday Y, Rather GM, Begh GR (2019) SINR analysis and interference management of macrocell cellular networks in dense urban environments. Wirel Pers Commun 111:1–21 Banday Y, Rather GM, Begh GR (2019) SINR analysis and interference management of macrocell cellular networks in dense urban environments. Wirel Pers Commun 111:1–21
41.
Zurück zum Zitat Guo S, Xiao B, Yang Y, et al. (2016) Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. In: IEEE INFOCOM 2016-the 35th annual IEEE International conference on computer communications. Washington: IEEE Computer Society Press, pp 1–9 Guo S, Xiao B, Yang Y, et al. (2016) Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. In: IEEE INFOCOM 2016-the 35th annual IEEE International conference on computer communications. Washington: IEEE Computer Society Press, pp 1–9
42.
Zurück zum Zitat Chen L, Qu H, Zhao J et al (2016) Efficient and robust deep learning with correntropy-induced loss function. Neural Comput Appl 27(4):1019–1031CrossRef Chen L, Qu H, Zhao J et al (2016) Efficient and robust deep learning with correntropy-induced loss function. Neural Comput Appl 27(4):1019–1031CrossRef
43.
Zurück zum Zitat Liu W, Anguelov D, Erhan D, et al. (2016) Ssd: single shot multibox detector. European conference on computer vision. Berlin: Springer, pp 21–37 Liu W, Anguelov D, Erhan D, et al. (2016) Ssd: single shot multibox detector. European conference on computer vision. Berlin: Springer, pp 21–37
44.
Zurück zum Zitat Chen J, Chen S, Wang Q et al (2019) iRAF: a deep reinforcement learning approach for collaborative mobile edge computing IoT networks. IEEE Internet Things J 6(4):7011–7024MathSciNetCrossRef Chen J, Chen S, Wang Q et al (2019) iRAF: a deep reinforcement learning approach for collaborative mobile edge computing IoT networks. IEEE Internet Things J 6(4):7011–7024MathSciNetCrossRef
45.
Zurück zum Zitat Dai H, Zeng X, Yu Z et al (2019) A scheduling algorithm for autonomous driving tasks on mobile edge computing servers. J Syst Architect 94:14–23CrossRef Dai H, Zeng X, Yu Z et al (2019) A scheduling algorithm for autonomous driving tasks on mobile edge computing servers. J Syst Architect 94:14–23CrossRef
46.
Zurück zum Zitat Chen M, Hao Y (2018) Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J Sel Areas Commun 36(3):587–597CrossRef Chen M, Hao Y (2018) Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J Sel Areas Commun 36(3):587–597CrossRef
47.
Zurück zum Zitat Guo H, Zhang J, Liu J et al (2018) Energy-aware computation offloading and transmit power allocation in ultradense IoT networks. IEEE Internet Things J 6(3):4317–4329CrossRef Guo H, Zhang J, Liu J et al (2018) Energy-aware computation offloading and transmit power allocation in ultradense IoT networks. IEEE Internet Things J 6(3):4317–4329CrossRef
48.
Zurück zum Zitat Rajule N, Ambudkar B, Dhande A (2013) Survey of vertical handover decision algorithms. Int J Innov Eng Technol 2(1):362–368 Rajule N, Ambudkar B, Dhande A (2013) Survey of vertical handover decision algorithms. Int J Innov Eng Technol 2(1):362–368
49.
Zurück zum Zitat Larasati HT, Hakimi R, Juhana T (2017) Extended-LLF: a least loaded first (LLF)-based handover association control for software-defined wireless network. Int J Comput Eng Inf Technol 9(9):203 Larasati HT, Hakimi R, Juhana T (2017) Extended-LLF: a least loaded first (LLF)-based handover association control for software-defined wireless network. Int J Comput Eng Inf Technol 9(9):203
50.
Zurück zum Zitat Goutam S, Unnikrishnan S (2019) QoS based vertical handover decision algorithm using fuzzy logic. In: 2019 international conference on nascent technologies in engineering. Washington: IEEE Computer Society Press, pp 1–7 Goutam S, Unnikrishnan S (2019) QoS based vertical handover decision algorithm using fuzzy logic. In: 2019 international conference on nascent technologies in engineering. Washington: IEEE Computer Society Press, pp 1–7
51.
Zurück zum Zitat Kobayashi R, Adachi K (2019) Radio and computing resource allocation for minimizing total processing completion time in mobile edge computing. IEEE Access 7:141119–141132CrossRef Kobayashi R, Adachi K (2019) Radio and computing resource allocation for minimizing total processing completion time in mobile edge computing. IEEE Access 7:141119–141132CrossRef
52.
Zurück zum Zitat Nguyen PD, Ha VN, Le LB (2019) Computation offloading and resource allocation for backhaul limited cooperative MEC systems. In: 2019 IEEE 90th vehicular technology conference. Washington: IEEE Computer Society Press, pp 1–6 Nguyen PD, Ha VN, Le LB (2019) Computation offloading and resource allocation for backhaul limited cooperative MEC systems. In: 2019 IEEE 90th vehicular technology conference. Washington: IEEE Computer Society Press, pp 1–6
Metadaten
Titel
Deep reinforcement learning-based resource allocation and seamless handover in multi-access edge computing based on SDN
verfasst von
Chunlin Li
Yong Zhang
Youlong Luo
Publikationsdatum
21.07.2021
Verlag
Springer London
Erschienen in
Knowledge and Information Systems / Ausgabe 9/2021
Print ISSN: 0219-1377
Elektronische ISSN: 0219-3116
DOI
https://doi.org/10.1007/s10115-021-01590-4

Weitere Artikel der Ausgabe 9/2021

Knowledge and Information Systems 9/2021 Zur Ausgabe

Premium Partner