Skip to main content
Top
Published in: Wireless Networks 7/2022

08-07-2022 | Original Paper

DDPG-based intelligent rechargeable fog computation offloading for IoT

Authors: Siguang Chen, Xinwei Ge, Qian Wang, Yifeng Miao, Xiukai Ruan

Published in: Wireless Networks | Issue 7/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In view of the existing computation offloading research on fog computing network scenarios, most scenarios focus on reducing energy consumption and delay and lack the joint consideration of smart device rechargeability. This paper proposes a deep deterministic policy gradient-based intelligent rechargeable fog computation offloading mechanism that is combined with simultaneous wireless information and power transfer technology. Specifically, an optimization problem that minimizes the total energy consumption for completing all tasks in a multiuser scenario is formulated, and the joint optimization of the task offloading ratio, uplink channel bandwidth, power split ratio and computing resource allocation is fully considered. Based on the above nonconvex optimization problem with a continuous action space, a communication, computation and energy harvesting co-aware intelligent computation offloading algorithm is developed. It can achieve the optimal energy consumption and delay, and similar to a double deep Q-network, an inverting gradient updating-based dual actor-critic neural network design can improve the convergence and stability of the training process. Finally, the simulation results validate that the proposed mechanism can converge quickly and can effectively reduce the energy consumption with the lowest task delay.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Cao, B., Li, Y., Zhang, L., Zhang, L., Mumtaz, S., Zhou, Z., & Peng, M. (2019). When Internet of Things meets blockchain: Challenges in distributed consensus. IEEE Network, 33(6), 133–139.CrossRef Cao, B., Li, Y., Zhang, L., Zhang, L., Mumtaz, S., Zhou, Z., & Peng, M. (2019). When Internet of Things meets blockchain: Challenges in distributed consensus. IEEE Network, 33(6), 133–139.CrossRef
2.
go back to reference Zhao, J., Li, Q., Gong, Y., & Zhang, K. (2019). Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks. IEEE Transactions on Vehicular Technology, 68(8), 7944–7956.CrossRef Zhao, J., Li, Q., Gong, Y., & Zhang, K. (2019). Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks. IEEE Transactions on Vehicular Technology, 68(8), 7944–7956.CrossRef
3.
go back to reference Cao, B., Zhang, L., Li, Y., Feng, D., & Cao, W. (2019). Intelligent offloading in multi-access edge computing: A state-of-the-art review and framework. IEEE Communications Magazine, 57(3), 56–62.CrossRef Cao, B., Zhang, L., Li, Y., Feng, D., & Cao, W. (2019). Intelligent offloading in multi-access edge computing: A state-of-the-art review and framework. IEEE Communications Magazine, 57(3), 56–62.CrossRef
4.
go back to reference Chen, L., Zhou, P., Gao, L., & Xu, J. (2018). Adaptive fog configuration for the industrial Internet of Things. IEEE Transactions on Industrial Informatics, 14(10), 4656–4664.CrossRef Chen, L., Zhou, P., Gao, L., & Xu, J. (2018). Adaptive fog configuration for the industrial Internet of Things. IEEE Transactions on Industrial Informatics, 14(10), 4656–4664.CrossRef
5.
go back to reference Xiang, H., Peng, M., Sun, Y., & Yan, S. (2020). Mode selection and resource allocation in sliced fog radio access networks: A reinforcement learning approach. IEEE Transactions on Vehicular Technology, 69(4), 4271–4284.CrossRef Xiang, H., Peng, M., Sun, Y., & Yan, S. (2020). Mode selection and resource allocation in sliced fog radio access networks: A reinforcement learning approach. IEEE Transactions on Vehicular Technology, 69(4), 4271–4284.CrossRef
6.
go back to reference Fang, F., Wang, K., Ding, Z., & Leung, V. C. (2021). Energy-efficient resource allocation for NOMA-MEC networks with imperfect CSI. IEEE Transactions on Communications, 69(5), 3436–3449.CrossRef Fang, F., Wang, K., Ding, Z., & Leung, V. C. (2021). Energy-efficient resource allocation for NOMA-MEC networks with imperfect CSI. IEEE Transactions on Communications, 69(5), 3436–3449.CrossRef
7.
go back to reference Liu, L., Chang, Z., Guo, X., Mao, S., & Ristaniemi, T. (2018). Multiobjective optimization for computation offloading in fog computing. IEEE Internet of Things Journal, 5(1), 283–294.CrossRef Liu, L., Chang, Z., Guo, X., Mao, S., & Ristaniemi, T. (2018). Multiobjective optimization for computation offloading in fog computing. IEEE Internet of Things Journal, 5(1), 283–294.CrossRef
8.
go back to reference Zhao, Z., Bu, S., Zhao, T., Yin, Z., Peng, M., Ding, Z., & Quek, T. Q. (2019). On the design of computation offloading in fog radio access networks. IEEE Transactions on Vehicular Technology, 68(7), 7136–7149.CrossRef Zhao, Z., Bu, S., Zhao, T., Yin, Z., Peng, M., Ding, Z., & Quek, T. Q. (2019). On the design of computation offloading in fog radio access networks. IEEE Transactions on Vehicular Technology, 68(7), 7136–7149.CrossRef
9.
go back to reference Zhang, L., Cao, B., Li, Y., Peng, M., & Feng, G. (2021). A multi-stage stochastic programming-based offloading policy for fog enabled IoT-eHealth. IEEE Journal on Selected Areas in Communications, 39(2), 411–425.CrossRef Zhang, L., Cao, B., Li, Y., Peng, M., & Feng, G. (2021). A multi-stage stochastic programming-based offloading policy for fog enabled IoT-eHealth. IEEE Journal on Selected Areas in Communications, 39(2), 411–425.CrossRef
10.
go back to reference Liu, Y., Yu, F. R., Li, X., Ji, H., & Leung, V. C. (2018). Distributed resource allocation and computation offloading in fog and cloud networks with non-orthogonal multiple access. IEEE Transactions on Vehicular Technology, 67(12), 12137–12151.CrossRef Liu, Y., Yu, F. R., Li, X., Ji, H., & Leung, V. C. (2018). Distributed resource allocation and computation offloading in fog and cloud networks with non-orthogonal multiple access. IEEE Transactions on Vehicular Technology, 67(12), 12137–12151.CrossRef
11.
go back to reference Chen, S., You, Z., & Ruan, X. (2020). Privacy and energy co-aware data aggregation computation offloading for fog-assisted IoT networks. IEEE Access, 8, 72424–72434.CrossRef Chen, S., You, Z., & Ruan, X. (2020). Privacy and energy co-aware data aggregation computation offloading for fog-assisted IoT networks. IEEE Access, 8, 72424–72434.CrossRef
12.
go back to reference Tan, L., Hu, R., & Hanzo, L. (2019). Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks. IEEE Transactions on Vehicular Technology, 68(4), 3086–3099.CrossRef Tan, L., Hu, R., & Hanzo, L. (2019). Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks. IEEE Transactions on Vehicular Technology, 68(4), 3086–3099.CrossRef
13.
go back to reference Wei, Y., Yu, F. R., Song, M., & Han, Z. (2019). Joint optimization of caching, computing, and radio resources for fog-enabled IoT using natural actor-critic deep reinforcement learning. IEEE Internet of Things Journal, 6(2), 2061–2073.CrossRef Wei, Y., Yu, F. R., Song, M., & Han, Z. (2019). Joint optimization of caching, computing, and radio resources for fog-enabled IoT using natural actor-critic deep reinforcement learning. IEEE Internet of Things Journal, 6(2), 2061–2073.CrossRef
14.
go back to reference Volodymyr, M., Koray, K., David, S., Rusu, A. A., Veness, J., Bellemare, M. G., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.CrossRef Volodymyr, M., Koray, K., David, S., Rusu, A. A., Veness, J., Bellemare, M. G., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.CrossRef
15.
go back to reference Wu, C., Yoshinaga, T., Ji, Y., Murase, T., & Zhang, Y. (2017). A reinforcement learning-based data storage scheme for vehicular ad hoc networks. IEEE Transactions on Vehicular Technology, 66(7), 6336–6348.CrossRef Wu, C., Yoshinaga, T., Ji, Y., Murase, T., & Zhang, Y. (2017). A reinforcement learning-based data storage scheme for vehicular ad hoc networks. IEEE Transactions on Vehicular Technology, 66(7), 6336–6348.CrossRef
16.
go back to reference Lei, L., Xu, H., Xiong, X., Zheng, K., Xiang, W., & Wang, X. (2019). Multi-user resource control with deep reinforcement learning in IoT edge computing. IEEE Internet of Things Journal, 6(6), 10119–10133.CrossRef Lei, L., Xu, H., Xiong, X., Zheng, K., Xiang, W., & Wang, X. (2019). Multi-user resource control with deep reinforcement learning in IoT edge computing. IEEE Internet of Things Journal, 6(6), 10119–10133.CrossRef
17.
go back to reference Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019). Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal, 6(3), 4005–4018.CrossRef Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019). Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal, 6(3), 4005–4018.CrossRef
18.
go back to reference Huang, L., Bi, S., & Zhang, Y. (2020). Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Transactions on Mobile Computing, 19(11), 2581–2593.CrossRef Huang, L., Bi, S., & Zhang, Y. (2020). Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Transactions on Mobile Computing, 19(11), 2581–2593.CrossRef
19.
go back to reference Liu, Y., Yu, H., Xie, S., & Zhang, Y. (2019). Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks. IEEE Transactions on Vehicular Technology, 68(11), 11158–11168.CrossRef Liu, Y., Yu, H., Xie, S., & Zhang, Y. (2019). Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks. IEEE Transactions on Vehicular Technology, 68(11), 11158–11168.CrossRef
20.
go back to reference Chen, S., Chen, J., & Zhao, C. (2021). Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism. Tien Tzu Hsueh Pao, 49(1), 157–166. Chen, S., Chen, J., & Zhao, C. (2021). Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism. Tien Tzu Hsueh Pao, 49(1), 157–166.
21.
go back to reference Zhou, F., & Hu, R. Q. (2020). Computation efficiency maximization in wireless-powered mobile edge computing networks. IEEE Transactions on Wireless Communications, 19(5), 3170–3184.CrossRef Zhou, F., & Hu, R. Q. (2020). Computation efficiency maximization in wireless-powered mobile edge computing networks. IEEE Transactions on Wireless Communications, 19(5), 3170–3184.CrossRef
22.
go back to reference Chen, S., Zheng, Y., Wang, K., & Lu, W. (2019). Delay guaranteed energy-efficient computation offloading for industrial IoT in fog computing. In Proceedings of the IEEE international conference on communications (ICC) (pp. 1–6). Chen, S., Zheng, Y., Wang, K., & Lu, W. (2019). Delay guaranteed energy-efficient computation offloading for industrial IoT in fog computing. In Proceedings of the IEEE international conference on communications (ICC) (pp. 1–6).
Metadata
Title
DDPG-based intelligent rechargeable fog computation offloading for IoT
Authors
Siguang Chen
Xinwei Ge
Qian Wang
Yifeng Miao
Xiukai Ruan
Publication date
08-07-2022
Publisher
Springer US
Published in
Wireless Networks / Issue 7/2022
Print ISSN: 1022-0038
Electronic ISSN: 1572-8196
DOI
https://doi.org/10.1007/s11276-022-03054-1

Other articles of this Issue 7/2022

Wireless Networks 7/2022 Go to the issue