Top

Peer-to-Peer Networking and Applications

Published in:

16-08-2021

Double agents-DQL based D2D computing-offloading for SHVC

Authors: Jianlong Liu, Jiaye Wen, Lixia Lin, Wen’an Zhou

Published in: Peer-to-Peer Networking and Applications | Issue 1/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

To make the network provide a better Quality-of-Service (QoS) guarantee for compute-intensive applications, many researchers study Device-to-Device (D2D) based Multi-access Edge Computing (MEC) for reducing computing task completion time and energy consumption. However, the majority of works applies only to the scenario, in which, a specific video coding algorithm with a fixed number of enhancement layer is employed, i.e., the computing task with certain computing amount. Then, the computing task can only be continuously executed by edge computing-offloading. In fact, the Scalable High-efficiency Video Coding (SHVC) algorithm has an uncertain computing amount. Also, the computing task has multiple execution steps by D2D computing-offloading. This is because the executor may interrupt the execution of the ongoing SHVC computing task and change its execution step. Therefore, it is a challenging problem that, how to minimize the completion time and energy consumption of this scenario. To solve this problem, firstly, we model the SHVC computing tasks by ten atomic operations. Secondly, we propose an action space dynamic generation method to solve the reward sparse problem, and we propose an action space coding method to reduce the storage space and searching cost of experience pool matrix, and we employ a double-agent deep Q-learning algorithm to further improve the efficiency of solving the problem. Finally, the simulation results show that the long-term QoS of users is improved.

previous article A novel hierarchical fault management framework for wireless sensor networks: HFMF

next article Enabling rank-based distribution of microservices among containers for green cloud computing environment

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Cisco (2019) Cisco visual networking index: global Mobile data traffic forecast update 2017–2022. Cisco, San Jose, CA, USA

Fatemidokht H, Rafsanjani, MK, Gupta BB, Hsu C (2021) Efficient and secure routing protocol based on artificial intelligence algorithms with UAV-assisted for vehicular ad hoc networks in intelligent transportation systems. IEEE Transactions on Intelligent Transportation Systems, Early Access:1–13

Mirsadeghi F, Rafsanjani MK, Gupta BB (2020) A trust infrastructure-based authentication method for clustered vehicular ad hoc networks. Peer-to-Peer Netw Appl. https://doi.org/10.1007/s12083-020-01010-4

Esposito C, Ficco M, Gupta BB (2021) Blockchain-based authentication and authorization for smart city applications. Inf Process Manag 58(2):1–16CrossRef

Li D, Deng L, Gupta BB, Wang H, Choi C (2019) A novel CNN based security guaranteed image watermarking generation scenario for smart city applications. Inf Sci 479:432–447CrossRef

Choi C, Wang T, Esposito C, Gupta BB, Lee K (2021) Sensored semantic annotation for traffic control based on knowledge inference in video. IEEE Sensors J 21(10):11758–11768CrossRef

Abbas N, Zhang Y, Taherkordi A, Skeie T (2018) Mobile edge computing: a survey. IEEE Internet Things J 5(1):450–460CrossRef

Wang S, Zhang X, Zhang Y, Lin W, Yang J, Wang W (2017) A survey on Mobile edge networks: convergence of computing, caching and communications. IEEE Access 5:6757–6779CrossRef

Chatzopoulos D, Bermejo C, ul Haq E, Li Y, Hui P (2019) D2D task offloading: a dataset-based Q&a. IEEE Commun Mag 57(2):102–107CrossRef

10.

Zhu Z, Liu T, Yang Y, Luo X (2019) BLOT: bandit learning-based offloading of tasks in fog-enabled networks. IEEE Trans Parallel Distrib Syst 30(12):2636–2649CrossRef

11.

Liu Y, Liu J, Argyriou A, Ci S (2019) MEC-assisted panoramic VR video streaming over millimeter wave Mobile networks. IEEE Trans Multimedia 21(5):1302–1316CrossRef

12.

He G, Hu J, Jiang H, Li Y (2018) Scalable video coding based on user’s view for real-time virtual reality applications. IEEE Commun Lett 22(1):25–28CrossRef

13.

Naeem RZ, Bashir S, Amjad MF, Abbas H, Afzal H (2019) Fog computing in internet of things: practical applications and future directions. Peer-to-Peer Netw Appl 12:1236–1262CrossRef

14.

Yao C, Wang X, Zheng Z, Sun G, Song L (2018) EdgeFlow: open-source multi-layer data flow processing in edge computing for 5G and beyond. IEEE Netw 33(2):166–173CrossRef

15.

Kumar N, Vikas P, Gupta BB, Goyal MK (2020) A novel framework for risk assessment and resilience of critical infrastructure towards climate change. Technol Forecast Soc Change 165:1–12

16.

Zhang F, Liu G, Zhao B, Xiaoming F, Yahyapour R (2019) Reducing the network overhead of user mobility-induced virtual machine migration in mobile edge computing. Software-Practice & Experience 49:673–693CrossRef

17.

Gupta BB, Quamara M (2020) An overview of internet of things (IoT): architectural aspects, challenges, and protocols. Concurr Comput Pract Exp 32(21):1–12CrossRef

18.

Cao B, Zhang L, Li Y, Feng D, Cao W (2019) Intelligent offloading in multi-access edge computing: a state-of-the-art review and framework. IEEE Commun Mag 57(3):56–62CrossRef

19.

Yuan P, Huang R (2021) Integrating the device-to-device communication technology into edge computing: a case study. Peer-to-Peer Netw Appl 14:599–608CrossRef

20.

Long C, Yang C, Jiang T, Zhang Q (2018) Edge computing framework for cooperative video processing in multimedia IoT systems. IEEE Trans Multimedia 20(5):1126–1139CrossRef

21.

Shah SS, Ali M, Malik AW, Khan MA, Ravana SD (2019) vFog: a vehicle-assisted computing framework for delay-sensitive applications in smart cities. IEEE Access 7:34900–34909CrossRef

22.

Sun J, Qing G, Zheng T, Dong P, Qin Y (2019) Joint communication and computing resource allocation in vehicular edge computing. Int J Distrib Sens Netw 15(3):1–13CrossRef

23.

Guo H, Zhang J, Liu J (2019) FiWi-enhanced vehicular edge computing networks: collaborative computation task offloading. IEEE Veh Technol Mag 14(1):45–53CrossRef

24.

Chen W, Liu B, Huang H, Guo S, Zheng Z (2019) When UAV swarm meets edge-cloud computing: the QoS perspective. IEEE Netw 33(2):36–43CrossRef

25.

Sun Y, Guo X, Song J, Zhou S, Jiang Z, Liu X, Niu Z (2019) Adaptive learning-based task offloading for vehicular edge computing systems. IEEE Trans Veh Technol 68(4):3061–3074CrossRef

26.

Xu C, Lingjun P, Lin G, Wu W, Wu D (2017) Exploiting massive D2D collaboration for energy-efficient Mobile edge computing. IEEE Wirel Commun 24(4):64–71CrossRef

27.

Yan H, Li Y, Zhu X, Zhang D, Wang J, Chen H, Bao W (2019) EASE: energy-efficient task scheduling for edge computing under uncertain runtime and unstable communication conditions. Concurr Comput Pract Exp 33(7):1–19CrossRef

28.

Tang M, Lin G, Huang J (2018) Enabling edge cooperation in tactile internet via 3C resource sharing. IEEE J Sel Areas Commun 36(11):2444–2454CrossRef

29.

Li S, Tao Y, Qin X, Liu L, Zhang Z, Zhang P (2019) Energy-aware Mobile edge computation offloading for IoT over Heterogenous networks. IEEE Access 7:13092–13105CrossRef

30.

He X, Xing H, Chen Y, Nallanathan A (2019) Energy-Efficient Mobile-Edge Computation Offloading for Applications with Shared Data. IEEE Global Communications Conference, Abu Dhabi, United Arab Emirates, 1–6, Dec 9–13

31.

Selim MM, Rihan M, Yang Y, Ma J (2021) Optimal task partitioning, bit allocation and trajectory for D2D-assisted UAV-MEC systems. Peer-to-Peer Netw Appl 14:215–224CrossRef

32.

Jia Q, Xie R, Tang Q, Li X, Huang T, Liu J, Liu Y (2019) Energy-efficient computation offloading in 5G cellular networks with edge computing and D2D communications. IET Communn 13(8):1122–1130CrossRef

33.

Lingjun P, Xu C, Mao G, Xie Q, Xu J (2019) Chimera: an energy-efficient and deadline-aware hybrid edge computing framework for vehicular Crowdsensing applications. IEEE Internet Things J 6(1):84–99CrossRef

34.

Zhou Y, Yang B, Wu C, Ren J, Zhang Y (2019) Mobile transparent computing: a novel user-centric approach to unify device, edge, and cloud. IEEE Netw 33(2):132–137CrossRef

35.

Wen J, Ren C, Sangaiah AK (2018) Energy-efficient device-to-device edge computing network: an approach offloading both traffic and computation. IEEE Commun Mag 56(9):96–102CrossRef

36.

Diao X, Zheng J, Wu Y, Cai Y (2019) Joint computing resource, power, and channel allocations for D2D-assisted and NOMA-based Mobile edge computing. IEEE Access 7:9243–9257CrossRef

37.

Golkarifard M, Yang J, Huang Z, Movaghar A, Pan H (2019) Dandelion: a unified code offloading system for wearable computing. IEEE Trans Mob Comput 18(3):546–559CrossRef

38.

Shao X, Hasegawa G, Kamiyama N, Liu Z, Masui H, Ji Y (2019) Joint Optimization of Computing Resources and Data Allocation for Moble Edge Computing (MEC): An Online Approach, International Conference on Computer Communication and Networks. Valencia, Spain, 1–9, Jul 29-Aug 1

39.

Yang J, Luo J, Meng D, Hwang J-N (2019) QoE-driven resource allocation optimized for uplink delivery of delay-sensitive VR video over cellular network. IEEE Access 7:60672–60683CrossRef

40.

Guo F, Richard Yu F, Zhang H, Ji H, Leung VCM, Li X (2020) An adaptive wireless virtual reality framework in future wireless networks: a distributed learning approach. IEEE Trans Veh Technol 69(8):8514–8528CrossRef

41.

Sun Y, Chen Z, Tao M, Liu H (2018) Communication, Computing and Caching for Mobile VR Delivery: Modeling and Trade-Off. IEEE International Conference on Communications, Kansas City, MO, USA, 1–6, May 20–24

42.

Zheng C, Liu S, Huang Y, Yang L (2020) MEC-Enabled Wireless VR Video Service: A Learning-Based Mixed Strategy for Energy-Latency Tradeoff. IEEE Wireless Communications and Networking Conference, Seoul, South Korea, 1–6, May 25–28

43.

Jianbo D, Richard Yu F, Lu G, Wang J, Jiang J, Chu X (2020) MEC-assisted immersive VR video streaming over terahertz wireless networks: a deep reinforcement learning approach. IEEE Internet Things J 7(10):9517–9529CrossRef

44.

Zhang W, Wen Y, Guan K, Kilper D, Luo H, Wu DO (2013) Energy-optimal Mobile cloud computing under stochastic Wireless Channel. IEEE Trans Wirel Commun 12(9):4569–4581CrossRef

45.

Rodrigues TK, Suto K, Nishiyama H, Liu J, Kato N (2019) Machine learning meets computation and communication control in evolving edge and cloud: challenges and future perspective. IEEE Commun Surv Tutor 22(1):38–67CrossRef

46.

Chen X, Zhang H, Wu C, Mao S, Ji Y, Bennis M (2019) Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet Things J 6(3):4005–4018CrossRef

47.

Ma X-Y, Sun Y-Z, Fang H-L, Tian Y (2013) Scenario-based multi-objective decision-making of optimal access point for wind power transmission corridor in the load centers. IEEE Trans Sustain Energy 4(1):229–239CrossRef

48.

Xu Z, Liao H (2014) Intuitionistic fuzzy analytic hierarchy process. IEEE Trans Fuzzy Syst 22(4):749–761CrossRef

49.

Yang F, Zhu Z, Zhao S, Yang Y, Luo X (2018) Optimal Task Offloading in Fog-Enabled Networks Via Index Policies. IEEE Global Conference on Signal & Information Processing, Anaheim, CA, USA, 1–5, Nov 26–29

50.

Yu S, Langar R, Wang X (2016) A D2D-Multicast Based Computation Offloading Framework for Interactive Applications. IEEE Global Communications Conference, Washington, DC, USA, 1–6, Dec 4–8

51.

Tao Y, You C, Zhang P, Huang K (2019) Stochastic control of computation offloading to a helper with a dynamically loaded CPU. IEEE Trans Wirel Commun 18(2):1247–1262CrossRef

52.

Li Z, Xie D, Pei T, Choi Y, Li R (2012) A Graph-Based Multi-hop Cooperative MIMO Scheme for Heterogeneous WSN. China Conference on Wireless Sensor Networks: Advances in Wireless Sensor Networks, Springer Berlin Heidelberg, 237–247, Oct 25–27

53.

Huang H, Liu B, Chen L, Xiang W, Hu M, Yang T (2018) D2D-assisted VR video pre-caching strategy. IEEE Access 6:61886–61895CrossRef

54.

Liu J, Luo K, Zhou Z, Xu C (2019) ERP: edge resource pooling for data stream Mobile computing. IEEE Internet Things J 6(3):4355–4368CrossRef

55.

Yang J, Si P, Wang Z, Jiang X, Hanzo L (2018) Dynamic resource allocation and layer selection for scalable video streaming in femtocell networks: a twin-time-scale approach. IEEE Trans Commun 66(8):3455–3470CrossRef

56.

Wu K, Jiang M, Tan H-Z (2018) D2D relay selection based on joint fuzzy and entropy theories with social similarity. IEEE Trans Veh Technol 67(9):8796–8807CrossRef

57.

Lu T, Dai H, Wang B (2018) QoE-Orientated Resource Allocation for Wireless VR over Small Cell Networks. International Conference on Wireless Communications and Signal Processing, Hangzhou, China, 1–6, Oct 1

58.

Liang H, Xu F, Cheng Z, Qian L, Wu Y (2019) Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing. Digit Commun Netw 5(1):10–17CrossRef

59.

Cermak G, Pinson M, Wolf S (2011) The relationship among video quality, screen resolution, and bit rate. IEEE Trans Broadcast 57(2):258–262CrossRef

60.

Brualdi RA (2009) Combinatorial mathematics, 5th edn. Pearson

Title: Double agents-DQL based D2D computing-offloading for SHVC
Authors: Jianlong Liu
Jiaye Wen
Lixia Lin
Wen’an Zhou
Publication date: 16-08-2021
Publisher: Springer US
Published in: Peer-to-Peer Networking and Applications / Issue 1/2022
Print ISSN: 1936-6442
Electronic ISSN: 1936-6450
DOI: https://doi.org/10.1007/s12083-021-01203-5

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2022

SCMP-V: A secure multiple relays cooperative downloading scheme with privacy preservation in VANETs

Energy aware MAC protocol with mobility management in wireless body area network

Robustness against fraudulent activities of a blockchain-based online review system

LocTrust: A local and global consensus-combined trust model in MANETs

Distributed Address Table (DAT): A Decentralized Model for End-to-End Communication in IoT

An autonomous intrusion detection system for the RPL protocol

Premium Partner