Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Wireless Networks
Published in: Wireless Networks 2/2021

02-01-2021

Uplink NOMA-based long-term throughput maximization scheme for cognitive radio networks: an actor–critic reinforcement learning approach

Authors: Hoang Thi Huong Giang, Tran Nhut Khai Hoan, Insoo Koo

Published in: Wireless Networks | Issue 2/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Non-orthogonal multiple access (NOMA) is one of the promising techniques for spectrum efficiency in wireless networks. In this paper, we consider an uplink NOMA cognitive system, where the secondary users (SUs) can jointly transmit data to the cognitive base station (CBS) over the same spectrum resources. Thereafter, successive interference cancellation is applied at the CBS to retrieve signals transmitted by the SUs. In addition, the energy-constrained problem in wireless networks is taken into account. Therefore, we assume that the SUs are powered by a wireless energy harvester to prolong their operations; meanwhile, the CBS is equipped with a traditional electrical supply. Herein, we propose an actor–critic reinforcement learning approach to maximize the long-term throughput of the cognitive network. In particular, by interacting and learning directly from the environment over several time slots, the CBS can optimally assign the amount of transmission energy for each SU according to the remaining energy of the SUs and the availability of the primary channel. As a consequence, the simulation results verify that the proposed scheme outperforms other conventional approaches (such as Myopic NOMA and OMA), so the system reward is always maximized in the current time slot, in terms of overall throughput and energy efficiency.
previous article Spectral efficiency (SE) enhancement of NOMA system through iterative power assignment
next article Cooperative link scheduling for RSU-assisted dissemination of basic safety messages

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Khan, F. A., Ratnarajah, T., & Sellathurai, M. (2010). Multiuser diversity analysis in spectrum sharing cognitive radio networks. In 2010 Proceedings of the fifth international conference on cognitive radio oriented wireless networks and communications, Cannes (pp. 1–5).
2.
Wang, C., et al. (2014). Cellular architecture and key technologies for 5G wireless communication networks. IEEE Communications Magazine, 52(2), 122–130.
3.
Hong, X., Wang, J., Wang, C., & Shi, J. (2014). Cognitive radio in 5G: A perspective on energy-spectral efficiency trade-off. IEEE Communications Magazine, 52(7), 46–53.
4.
Wang, B., & Liu, K. J. R. (2011). Advances in cognitive radio networks: A survey. IEEE Journal of Selected Topics in Signal Processing, 5(1), 5–23.
5.
Akyildiz, I. F., Lee, W., Vuran, M. C., & Mohanty, S. (2008). A survey on spectrum management in cognitive radio networks. IEEE Communications Magazine, 46(4), 40–48.
6.
Mitola, J, I. I. I., & Maguire, G. Q, Jr. (1999). Cognitive radio: Making software radios more personal. IEEE Personal Communications, 6(4), 13–18.
7.
Haykin, S. (2005). Cognitive radio: Brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications, 23(2), 201–220.
8.
Hossain, E., Niyato, D., & Han, Z. (2009). Dynamic spectrum access and management in cognitive radio networks. Cambridge: Cambridge University Press.
9.
Lv, L., Chen, J., Ni, A., Ding, Q. Z., & Jiang, H. (2018). Cognitive non-orthogonal multiple access with cooperative relaying: A new wireless frontier for 5G spectrum sharing. IEEE Communications Magazine, 56(4), 188–195.
10.
Goldsmith, A., Jafar, S. A., Maric, I., & Srinivasa, S. (2009). Breaking spectrum gridlock with cognitive radios: An information theoretic perspective. Proceedings of the IEEE, 97(5), 894–914.
11.
Giang, H. T. H., Hoan, T. N. K., Thanh, P. D., & Koo, I. (2019). A POMDP-based long-term transmission rate maximization for cognitive radio networks with wireless-powered ambient backscatter. International Journal of Communication Systems,. https://​doi.​org/​10.​1002/​dac.​3993.CrossRef
12.
Ding, Z., et al. (2017). Application of non-orthogonal multiple access in LTE and 5G networks. IEEE Communications Magazine, 55(2), 185–191.
13.
Dai, L., Wang, B., Yuan, Y., Han, S., Chih-Lin, I., & Wang, Z. (2015). Non-orthogonal multiple access for 5G: Solutions, challenges, opportunities, and future research trends. IEEE Communications Magazine, 53(9), 74–81.
14.
Wei, Z., Yuan, J., Ng, D. W. K., Elkashlan, M., & Ding, Z. (2016). A survey of downlink non-orthogonal multiple access for 5G wireless communication networks. ZTE Communications, 14(4), 17–26.
15.
Saito, Y., Kishiyama,Y., Benjebbour, A., Nakamura, T., Li, A., & Higuchi, K. (2013). Non-orthogonal multiple access (NOMA) for cellular future radio access. In 2013 IEEE 77th vehicular technology conference (VTC Spring), Dresden (pp. 1–5).
16.
Ding, Z., Peng, M., & Poor, H. V. (2015). Cooperative non-orthogonal multiple access in 5G systems. IEEE Communications Letters, 19(8), 1462–1465.
17.
Wan, D., Wen, M., Ji, F., Yu, H., & Chen, F. (2018). Non-orthogonal multiple access for cooperative communications: Challenges, opportunities, and trends. IEEE Wireless Communications, 25(2), 109–117.
18.
Tabassum, H., Hossain, E., & Hossain, J. (2017). Modeling and analysis of up-link non-orthogonal multiple access in large-scale cellular networks using Poisson cluster processes. IEEE Transactions on Communications, 65(8), 3555–3570.
19.
Razavi, R., Hoshyar, R., Imran, M. A., & Wang, Y. (2011). Information theoretic analysis of LDS scheme. IEEE Communications Letters, 15(8), 798–800.
20.
AL-Imari, M., Imran, M. A. C., & Tafazolli, R. (2012). Low Density Spreading for next generation multicarrier cellular systems. In 2012 International conference on future communication networks, Baghdad (pp. 52–57).
21.
Du, Y., Dong, B., Chen, Z., Fang, J. C., & Wang, X. (2016). A fast convergence multiuser detection scheme for uplink SCMA systems. IEEE Wireless Communications Letters, 5(4), 388–391.
22.
Nikopour, H., et al. (2014). SCMA for downlink multiple access of 5G wireless networks. In 2014 IEEE global communications conference, Austin, TX (pp. 3940–3945).
23.
Liu, Y., Ding, Z., Elkashlan, M., & Yuan, J. (2016). Nonorthogonal multiple access in large-scale underlay cognitive radio networks. IEEE Transactions on Vehicular Technology, 65(12), 10152–10157.
24.
Wang, D., & Men, S. (2018). Secure energy efficiency for NOMA based cognitive radio networks with nonlinear energy harvesting. IEEE Access, 6, 62707–62716.
25.
Lv, L., Ni, Q., Ding, Z., & Chen, J. (2017). Application of non-orthogonal multiple access in cooperative spectrum-sharing networks over Nakagami-\(m\) fading channels. IEEE Transactions on Vehicular Technology, 66(6), 5506–5511.
26.
Lv, L., Chen, J., Ni, Q., & Ding, Z. (2017). Design of cooperative non-orthogonal multicast cognitive multiple access for 5G systems: User scheduling and performance analysis. IEEE Transactions on Communications, 65(6), 2641–2656.
27.
Simjee, F. I., & Chou, P. H. (2008). Efficient charging of supercapacitors for extended lifetime of wireless sensor nodes. IEEE Transactions on Power Electronics, 23(3), 1526–1536.
28.
Chen, Z., Law, M., Mak, P., & Martins, R. P. (2017). A single-chip solar energy harvesting IC using integrated photodiodes for biomedical implant applications. IEEE Transactions on Biomedical Circuits and Systems, 11(1), 44–53.
29.
Wang, C., Li, J., Yang, Y., & Ye, F. (2018). Combining solar energy harvesting with wireless charging for hybrid wireless sensor networks. IEEE Transactions on Mobile Computing, 17(3), 560–576.
30.
Stuyts, J., Horn, G., Vandermeulen, W., Driesen, J., & Diehl, M. (2015). Effect of the electrical energy conversion on optimal cycles for pumping airborne wind energy. IEEE Transactions on Sustainable Energy, 6(1), 2–10.
31.
Zhao, L., Tang, L., Liang, J., & Yang, Y. (2017). Synergy of wind energy harvesting and synchronized switch harvesting interface circuit. IEEE/ASME Transactions on Mechatronics, 22(2), 1093–1103.
32.
Zou, Z., Gidmark, A., Charalambous, T., & Johansson, M. (2016). Optimal radio frequency energy harvesting with limited energy arrival knowledge. IEEE Journal on Selected Areas in Communications, 34(12), 3528–3539.
33.
Celik, A., Alsharoa, A., & Kamal, A. E. (2017). Hybrid energy harvesting-based cooperative spectrum sensing and access in heterogeneous cognitive radio networks. IEEE Transactions on Cognitive Communications and Networking, 3(1), 37–48.
34.
Zhu, L., Zhang, J., Xiao, Z., Cao, X., Wu, D. O., & Xia, X. (2018). Joint power control and beamforming for uplink non-orthogonal multiple access in 5G millimeter-wave communications. IEEE Transactions on Wireless Communications, 17(9), 6177–6189.
35.
Ali, M. S., Tabassum, H., & Hossain, E. (2016). Dynamic user clustering and power allocation for uplink and downlink non-orthogonal multiple access (NOMA) systems. IEEE Access, 4, 6325–6343.
36.
Zhai, D., & Du, J. (2018). Spectrum efficient resource management for multi-carrier-based NOMA networks: A graph-based method. IEEE Wireless Communications Letters, 7(3), 388–391.
37.
Thanh, P. D., Hoan, T. N. K., & Koo, I. (2020). Joint resource allocation and transmission mode selection using a POMDP-based hybrid half-duplex/full-duplex scheme for secrecy rate maximization in multi-channel cognitive radio networks. IEEE Sensors Journal, 20(7), 3930–3945.
38.
Shah, H. A., Koo, I., & Kyung, S. K. (2019). Actor–critic-algorithm-based accurate spectrum sensing and transmission framework and energy conservation in energy-constrained wireless sensor network-based cognitive radios. Wireless Communications and Mobile Computing,. https://​doi.​org/​10.​1155/​2019/​6051201.CrossRef
39.
Li, X., Fang, J., Cheng, W., Duan, H., Chen, Z., & Li, H. (2018). Intelligent power control for spectrum sharing in cognitive radios: A deep reinforcement learning approach. IEEE Access, 6, 25463–25473.
40.
Yang, H., & Xie, X. (2020). An actor–critic deep reinforcement learning approach for transmission scheduling in cognitive internet of things systems. IEEE Systems Journal, 14(1), 51–60.
41.
Zhang, Y., Wang, X. & Xu, Y. (2019). Energy-efficient resource allocation in uplink NOMA systems with deep reinforcement learning. In 2019 11th International conference on wireless communications and signal processing (WCSP), Xi’an, China (pp. 1–6).
42.
Zhang, J., Tao, X., Wu, H., Zhang, N., & Zhang, X. (2020). Deep reinforcement learning for throughput improvement of the uplink grant-free NOMA system. IEEE Internet of Things Journal, 7(7), 6369–6379.
43.
Manimekalai, T., Joan, S. R., & Laxmikandan, T. (2020). Throughput maximization for underlay CR multicarrier NOMA network with cooperative communication. ETRI Journal, 42(6), 846–858.
44.
Zhong, C., Lu, Z., Gursoy, M. C., & Velipasalar, S. (2018). Actor–critic deep reinforcement learning for dynamic multichannel access. In 2018 IEEE global conference on signal and information processing (GlobalSIP), Anaheim, CA, USA (pp. 599–603).
45.
Yang, Z., Feng, L., Chang, Z., Lu, J., Liu, R., Kadoch, M., & Cheriet, M. (2020). Prioritized uplink resource allocation in smart grid backscatter communication networks via deep reinforcement learning. Electronics, 9(4), 1–16.
46.
Yucek, T., & Arslan, H. (2009). A survey of spectrum sensing algorithms for cognitive radio applications. IEEE Communications Surveys & Tutorials, 11(1), 116–130.
47.
Cordeiro, C., & Challapali, K. (2005). Spectrum agile radios: Utilization and sensing architectures. In First IEEE international symposium on new frontiers in dynamic spectrum access networks (DySPAN 2005), Baltimore, MD, USA (pp. 160–169).
48.
Pawelczak, P., Janssen, G. J. M., & Prasad, R. V. (2006). WLC10-4: Performance measures of dynamic spectrum access networks. In IEEE Globecom 2006, San Francisco, CA (pp. 1–6).
49.
Liu, X., & Shankar, S. (2006). Sensing-based opportunistic channel access. Mobile Networks and Applications, 11(4), 577–591.
50.
Cichoń, K., Kliks, A., & Bogucka, H. (2016). Energy-efficient cooperative spectrum sensing: A survey. IEEE Communications Surveys & Tutorials, 18(3), 1861–1886.
51.
Quan, Z., Ma, W.-K., Cui, S., & Sayed, A. (2010). Optimal linear fusion for distributed detection via semidefinite programming. IEEE Transaction Signal Processing, 58(4), 2431–2436.MathSciNetMATH
52.
Ribeiro, F., de Campos, M., & Werner, S. (2012). Distributed cooperative spectrum sensing with adaptive combining. In 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), Kyoto, Japan (pp. 3557–3560).
53.
Han, W., Li, J., Li, Z., Si, J., & Zhang, Y. (2013). Efficient soft decision fusion rule in cooperative spectrum sensing. IEEE Transaction Signal Processing, 61(8), 1931–1943.MathSciNetMATH
54.
Stevenson, C. R., Chouinard, G., Lei, Z., Hu, W., Shellhammer, S. J., & Caldwell, W. (2009). IEEE 802.22 The first cognitive radio wireless regional area network standard. IEEE Communications Magazine, 47(1), 130–138.
55.
Liang, Y., Zeng, Y., Peh, E. C. Y., & Hoang, A. T. (2008). Sensing-throughput tradeoff for cognitive radio networks. IEEE Transactions on Wireless Communications, 7(4), 1326–1337.
56.
Crites, R. H., & Barto, A. G. (1995). An actor/critic algorithm that is equivalent to Q-learning. In Advances in neural information processing systems, Denver, CO (Vol. 7, pp. 401–408).
57.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge: MIT Press.MATH
58.
Singh, S., Jaakkola, T., Littman, M., & Szepesvri, C. (2000). Convergence results for single-step on-policy reinforcement-learning algorithms. Machine Learning, 38(3), 287–308.MATH
59.
Stone, J. V. (2013). Bayes’ rule: A tutorial introduction to Bayesian analysis. Sheffield: Sebtel Press.MATH
60.
Konda, V. R., & Tsitsiklis, J. N. (2000). Actor–critic algorithms. In Advances in neural information processing systems, CO (Vol. 12, pp. 1008–1014).
61.
Wang, K. (2018). Optimally myopic scheduling policy for downlink channels with imperfect state observation. IEEE Transactions on Vehicular Technology, 67(7), 5856–5867.
62.
Nguyen, T., Nguyen, V., Lee, J., & Kim, Y. (2019). Sum rate maximization for multi-user wireless powered IoT network with non-linear energy harvester: Time and power allocation. IEEE Access, 7, 149698–149710.
63.
Islam, S. M. R., Avazov, N., Dobre, O. A., & Kwak, K. (2017). Power-domain non-orthogonal multiple access (NOMA) in 5G systems: Potentials and challenges. IEEE Communications Surveys & Tutorials, 19(2), 721–742.
64.
Thanh, P. D., Hoan, T. N. K., Vu-Van, H., & Koo, I. (2019). Efficient attack strategy for legitimate energy-powered eavesdropping in tactical cognitive radio networks. Wireless Networks, 25(6), 3605–3622.
Metadata
Title
Uplink NOMA-based long-term throughput maximization scheme for cognitive radio networks: an actor–critic reinforcement learning approach
Authors
Hoang Thi Huong Giang
Tran Nhut Khai Hoan
Insoo Koo
Publication date
02-01-2021
Publisher
Springer US
Published in
Wireless Networks / Issue 2/2021
Print ISSN: 1022-0038
Electronic ISSN: 1572-8196
DOI
https://doi.org/10.1007/s11276-020-02520-y

Other articles of this Issue 2/2021

Wireless Networks 2/2021 Go to the issue

OriginalPaper

Threshold selection analysis of spectrum sensing for cognitive radio network with censoring based imperfect reporting channels

OriginalPaper

Efficient traffic control and lifetime maximization in mobile ad hoc network by using PSO–BAT optimization

OriginalPaper

Spectral efficiency (SE) enhancement of NOMA system through iterative power assignment

OriginalPaper

A comprehensive review of cooperative MIMO WSN: its challenges and the emerging technologies

Guest Editorial

Signal processing techniques for sustainable cognitive radio communications

OriginalPaper

Secure multiple target tracking based on clustering intersection points of measurement circles in wireless sensor networks