Skip to main content
Erschienen in: Pattern Analysis and Applications 1/2023

14.07.2022 | Theoretical Advances

User grouping and power allocation in NOMA systems: a novel semi-supervised reinforcement learning-based solution

verfasst von: Rebekka Olsson Omslandseter, Lei Jiao, Yuanwei Liu, B. John Oommen

Erschienen in: Pattern Analysis and Applications | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we present a pioneering solution to the problem of user grouping and power allocation in non-orthogonal multiple access (NOMA) systems. The problem is highly pertinent because NOMA is a well-recognized technique for future mobile radio systems. The salient and difficult issues associated with NOMA systems involve the task of grouping users together into the pre-specified time slots, which are augmented with the question of determining how much power should be allocated to the respective users. This problem is, in and of itself, NP-hard. Our solution is the first reported reinforcement learning (RL)-based solution, which attempts to resolve parts of this issue. In particular, we invoke the object migration automaton (OMA) and one of its variants to resolve the grouping in NOMA systems. Furthermore, unlike the solutions reported in the literature, we do not assume prior knowledge of the channels’ distributions, nor of their coefficients, to achieve the grouping/partitioning. Thereafter, we use the consequent groupings to heuristically infer the power allocation. The simulation results that we have obtained confirm that our learning scheme can follow the dynamics of the channel coefficients efficiently, and that the solution is able to resolve the issue dynamically.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
Object migration automata and its abbreviation, OMA, should not be confused with orthogonal multiple access, often also abbreviated OMA in the Literature. In this paper, the abbreviation OMA (and its variants) refers to the ML algorithm.
 
2
The abbreviation LA is used interchangeably throughout the paper, referring to both the field of learning automata and the learning automaton itself, depending on the context in which it appears.
 
3
The notation for the paper is given below in Table 1, so as to not distract from the content itself.
 
4
We assume that differences between the time average of channel coefficients are due to the distinct geolocations of the users. Although the channel coefficient for a user may vary in different frequency bands, it is assumed that the ranking of the time averages of the coefficients among the various users maintains the same order in the different bands due to their distinct geolocations. With this assumption, the heterogeneity in different frequency bands will not influence the results of the user grouping.
 
5
This assumption applies only in the problem formulation with instantaneous channel coefficients at \(t_0\). Understandably, the channel coefficients will change along time due to the stochastic behavior, and the ranking of instantaneous channel coefficients belonging to different users may change from time to time due to channel fading.
 
6
In reality, if \(L_c\) is not an integer, we can add dummy users to the system to satisfy the requirement. Dummy users are virtual users that are not part of the real network scenario, but are needed for constituting an equal size for all the clusters. Thus, the dummy users are used for the clustering, but these users are not real, and no resources are given to them in the power allocation process (they should be excluded in the power allocation process).
 
7
In these simulations, we used \(S=8\). The way that we obtained the solution’s accuracy was in terms of whether or not the EOMA found the clusters that it should have found, based on the mean values of the different users in the NOMA system.
 
Literatur
1.
Zurück zum Zitat Liu Y, Qin Z, Elkashlan M, Ding Z, Nallanathan A, Hanzo L (2017) Nonorthogonal multiple access for 5G and beyond. Proc IEEE 105(12):2347–2381CrossRef Liu Y, Qin Z, Elkashlan M, Ding Z, Nallanathan A, Hanzo L (2017) Nonorthogonal multiple access for 5G and beyond. Proc IEEE 105(12):2347–2381CrossRef
2.
Zurück zum Zitat Pischella M, Le Ruyet D (2019) NOMA-relevant clustering and resource allocation for proportional fair uplink communications. IEEE Wirel Commun Lett 8(3):873–876CrossRef Pischella M, Le Ruyet D (2019) NOMA-relevant clustering and resource allocation for proportional fair uplink communications. IEEE Wirel Commun Lett 8(3):873–876CrossRef
3.
Zurück zum Zitat Cui J, Ding Z, Fan P, Al-Dhahir N (2018) Unsupervised machine learning-based user clustering in millimeter-wave-NOMA systems. IEEE Trans Wirel Commun 17(11):7425–7440CrossRef Cui J, Ding Z, Fan P, Al-Dhahir N (2018) Unsupervised machine learning-based user clustering in millimeter-wave-NOMA systems. IEEE Trans Wirel Commun 17(11):7425–7440CrossRef
4.
Zurück zum Zitat Glimsdal S, Granmo O (2014) A novel Bayesian network based scheme for finding the optimal solution to stochastic online equi-partitioning problems. In: 13th International conference on machine learning and applications, pp 594–599 Glimsdal S, Granmo O (2014) A novel Bayesian network based scheme for finding the optimal solution to stochastic online equi-partitioning problems. In: 13th International conference on machine learning and applications, pp 594–599
5.
Zurück zum Zitat Gale W, Das S, Yu CT (1990) Improvements to an algorithm for equipartitioning. IEEE Trans Comput 39(5):706–710CrossRef Gale W, Das S, Yu CT (1990) Improvements to an algorithm for equipartitioning. IEEE Trans Comput 39(5):706–710CrossRef
6.
Zurück zum Zitat Shirvani A, Oommen BJ (2018) On invoking transitivity to enhance the pursuit-oriented object migration automata. IEEE Access 6:21668–21681CrossRef Shirvani A, Oommen BJ (2018) On invoking transitivity to enhance the pursuit-oriented object migration automata. IEEE Access 6:21668–21681CrossRef
7.
Zurück zum Zitat Shirvani A, Oommen BJ (2017) On utilizing the pursuit paradigm to enhance the deadlock-preventing object migration automaton. In: International conference on new trends in computing sciences (ICTCS), pp 295–302 Shirvani A, Oommen BJ (2017) On utilizing the pursuit paradigm to enhance the deadlock-preventing object migration automaton. In: International conference on new trends in computing sciences (ICTCS), pp 295–302
8.
Zurück zum Zitat Tsetlin ML (1973) Finite automata and modeling the simplest forms of behavior. In: Mathematics in science and engineering, vol 102, pp 3–83 Tsetlin ML (1973) Finite automata and modeling the simplest forms of behavior. In: Mathematics in science and engineering, vol 102, pp 3–83
9.
Zurück zum Zitat Lakshmivarahan S (1981) Learning algorithms: theory and applications. Springer, New YorkCrossRefMATH Lakshmivarahan S (1981) Learning algorithms: theory and applications. Springer, New YorkCrossRefMATH
10.
Zurück zum Zitat Narendra KS, Thathachar MAL (2013) Learning automata: an introduction. Courier Corporation, New york Narendra KS, Thathachar MAL (2013) Learning automata: an introduction. Courier Corporation, New york
11.
Zurück zum Zitat Lakshmivarahan S, Thathachar MAL (1973) Absolutely expedient algorithms for stochastic automata. IEEE Trans Syst Man Cybern 3:281–286MathSciNetMATH Lakshmivarahan S, Thathachar MAL (1973) Absolutely expedient algorithms for stochastic automata. IEEE Trans Syst Man Cybern 3:281–286MathSciNetMATH
12.
Zurück zum Zitat Oommen BJ, Agache M (2001) Continuous and discretized pursuit learning schemes: various algorithms and their comparison. IEEE Trans Syst Man Cybern B Cybern 31(3):277–287CrossRef Oommen BJ, Agache M (2001) Continuous and discretized pursuit learning schemes: various algorithms and their comparison. IEEE Trans Syst Man Cybern B Cybern 31(3):277–287CrossRef
13.
Zurück zum Zitat Zhang X, Granmo O-C, Oommen BJ (2012) Discretized Bayesian pursuit—a new scheme for reinforcement learning. In: Proceedings of IEA-AIE 2012, Dalian, China, pp 784–793 Zhang X, Granmo O-C, Oommen BJ (2012) Discretized Bayesian pursuit—a new scheme for reinforcement learning. In: Proceedings of IEA-AIE 2012, Dalian, China, pp 784–793
14.
Zurück zum Zitat Poznyak AS, Najim K (1997) Learning automata and stochastic optimization, vol 3. Springer, BerlinMATH Poznyak AS, Najim K (1997) Learning automata and stochastic optimization, vol 3. Springer, BerlinMATH
15.
Zurück zum Zitat Yazidi A, Zhang X, Jiao L, Oommen BJ (2020) The hierarchical continuous pursuit learning automation: a novel scheme for environments with large numbers of actions. IEEE Trans Neural Netw Learn Syst 31(2):512–526MathSciNetCrossRef Yazidi A, Zhang X, Jiao L, Oommen BJ (2020) The hierarchical continuous pursuit learning automation: a novel scheme for environments with large numbers of actions. IEEE Trans Neural Netw Learn Syst 31(2):512–526MathSciNetCrossRef
16.
Zurück zum Zitat Zhang X, Jiao L, Oommen BJ, Granmo O (2020) A conclusive analysis of the finite-time behavior of the discretized pursuit learning automaton. IEEE Trans Neural Netw Learn Syst 31(1):284–294MathSciNetCrossRef Zhang X, Jiao L, Oommen BJ, Granmo O (2020) A conclusive analysis of the finite-time behavior of the discretized pursuit learning automaton. IEEE Trans Neural Netw Learn Syst 31(1):284–294MathSciNetCrossRef
17.
Zurück zum Zitat Sedaghat MA, Müller RR (2018) On user pairing in uplink NOMA. IEEE Trans Wirel Commun 17(5):3474–3486CrossRef Sedaghat MA, Müller RR (2018) On user pairing in uplink NOMA. IEEE Trans Wirel Commun 17(5):3474–3486CrossRef
18.
Zurück zum Zitat Timotheou S, Krikidis I (2015) Fairness for non-orthogonal multiple access in 5G systems. IEEE Signal Process Lett 22(10):1647–1651CrossRef Timotheou S, Krikidis I (2015) Fairness for non-orthogonal multiple access in 5G systems. IEEE Signal Process Lett 22(10):1647–1651CrossRef
19.
Zurück zum Zitat Liu Y, Qin Z, Elkashlan M, Gao Y, Hanzo L (2017) Enhancing the physical layer security of non-orthogonal multiple access in large-scale networks. IEEE Trans Wirel Commun 16(3):1656–1672CrossRef Liu Y, Qin Z, Elkashlan M, Gao Y, Hanzo L (2017) Enhancing the physical layer security of non-orthogonal multiple access in large-scale networks. IEEE Trans Wirel Commun 16(3):1656–1672CrossRef
20.
Zurück zum Zitat Ding Z, Yang Z, Fan P, Poor HV (2014) On the performance of non-orthogonal multiple access in 5G systems with randomly deployed users. IEEE Signal Process Lett 21(12):1501–1505CrossRef Ding Z, Yang Z, Fan P, Poor HV (2014) On the performance of non-orthogonal multiple access in 5G systems with randomly deployed users. IEEE Signal Process Lett 21(12):1501–1505CrossRef
21.
Zurück zum Zitat Zhang N, Wang J, Kang G, Liu Y (2016) Uplink nonorthogonal multiple access in 5G systems. IEEE Commun Lett 20(3):458–461CrossRef Zhang N, Wang J, Kang G, Liu Y (2016) Uplink nonorthogonal multiple access in 5G systems. IEEE Commun Lett 20(3):458–461CrossRef
22.
Zurück zum Zitat Liu Y, Qin Z, Elkashlan M, Nallanathan A, McCann JA (2017) Non-orthogonal multiple access in large-scale heterogeneous networks. IEEE J Sel Areas Commun 35(12):2667–2680CrossRef Liu Y, Qin Z, Elkashlan M, Nallanathan A, McCann JA (2017) Non-orthogonal multiple access in large-scale heterogeneous networks. IEEE J Sel Areas Commun 35(12):2667–2680CrossRef
23.
Zurück zum Zitat Ding Z, Fan P, Poor HV (2017) Random beamforming in millimeter-wave NOMA networks. IEEE Access 5:7667–7681CrossRef Ding Z, Fan P, Poor HV (2017) Random beamforming in millimeter-wave NOMA networks. IEEE Access 5:7667–7681CrossRef
24.
Zurück zum Zitat Cui J, Liu Y, Ding Z, Fan P, Nallanathan A (2017) Optimal user scheduling and power allocation for millimeter wave NOMA systems. IEEE Trans Wirel Commun 17(3):1502–1517CrossRef Cui J, Liu Y, Ding Z, Fan P, Nallanathan A (2017) Optimal user scheduling and power allocation for millimeter wave NOMA systems. IEEE Trans Wirel Commun 17(3):1502–1517CrossRef
25.
Zurück zum Zitat Zhu L, Zhang J, Xiao Z, Cao X, Wu DO, Xia X (2018) Joint power allocation and beamforming for non-orthogonal multiple access (NOMA) in 5G millimeter wave communications. IEEE Trans Wirel Commun 17(9):6177–6189CrossRef Zhu L, Zhang J, Xiao Z, Cao X, Wu DO, Xia X (2018) Joint power allocation and beamforming for non-orthogonal multiple access (NOMA) in 5G millimeter wave communications. IEEE Trans Wirel Commun 17(9):6177–6189CrossRef
26.
Zurück zum Zitat Liu Y, Ding Z, Elkashlan M, Poor HV (2016) Cooperative non-orthogonal multiple access with simultaneous wireless information and power transfer. IEEE J Sel Areas Commun 34(4):938–953CrossRef Liu Y, Ding Z, Elkashlan M, Poor HV (2016) Cooperative non-orthogonal multiple access with simultaneous wireless information and power transfer. IEEE J Sel Areas Commun 34(4):938–953CrossRef
27.
Zurück zum Zitat Yin Y, Peng Y, Liu M, Yang J, Gui G (2019) Dynamic user grouping based NOMA over Rayleigh fading channels. IEEE Access 7:110964–110971CrossRef Yin Y, Peng Y, Liu M, Yang J, Gui G (2019) Dynamic user grouping based NOMA over Rayleigh fading channels. IEEE Access 7:110964–110971CrossRef
28.
Zurück zum Zitat Chen X, Benjebbour A, Li A, Harada A (2014) Multi-user proportional fair scheduling for uplink non-orthogonal multiple access (NOMA). In: IEEE 79th Vehicular technology conference (VTC Spring), pp 1–5 Chen X, Benjebbour A, Li A, Harada A (2014) Multi-user proportional fair scheduling for uplink non-orthogonal multiple access (NOMA). In: IEEE 79th Vehicular technology conference (VTC Spring), pp 1–5
29.
Zurück zum Zitat Liu F, Mähönen P, Petrova M (2015) Proportional fairness-based user pairing and power allocation for non-orthogonal multiple access. In: IEEE 26th Annual international symposium on personal, indoor, and mobile radio communications (PIMRC), pp 1127–1131 Liu F, Mähönen P, Petrova M (2015) Proportional fairness-based user pairing and power allocation for non-orthogonal multiple access. In: IEEE 26th Annual international symposium on personal, indoor, and mobile radio communications (PIMRC), pp 1127–1131
30.
Zurück zum Zitat Shirvani A (2018) Novel solutions and applications of the object partitioning problem. Ph.D. dissertation, Carleton University Shirvani A (2018) Novel solutions and applications of the object partitioning problem. Ph.D. dissertation, Carleton University
31.
Zurück zum Zitat Pätzold M (2012) Mobile radio channels, 2nd edn. Wiley, Chichester Pätzold M (2012) Mobile radio channels, 2nd edn. Wiley, Chichester
32.
Zurück zum Zitat Xing H, Liu Y, Nallanathan A, Ding Z, Poor HV (2018) Optimal throughput fairness tradeoffs for downlink non-orthogonal multiple access over fading channels. IEEE Trans Wirel Commun 17(6):3556–3571CrossRef Xing H, Liu Y, Nallanathan A, Ding Z, Poor HV (2018) Optimal throughput fairness tradeoffs for downlink non-orthogonal multiple access over fading channels. IEEE Trans Wirel Commun 17(6):3556–3571CrossRef
33.
Zurück zum Zitat Kang J, Kim I (2018) Optimal user grouping for downlink NOMA. IEEE Wirel Commun Lett 7(5):724–727CrossRef Kang J, Kim I (2018) Optimal user grouping for downlink NOMA. IEEE Wirel Commun Lett 7(5):724–727CrossRef
34.
Zurück zum Zitat Zhu L, Zhang J, Xiao Z, Cao X, Wu DO (2019) Optimal user pairing for downlink non-orthogonal multiple access (NOMA). IEEE Wirel Commun Lett 8(2):328–331CrossRef Zhu L, Zhang J, Xiao Z, Cao X, Wu DO (2019) Optimal user pairing for downlink non-orthogonal multiple access (NOMA). IEEE Wirel Commun Lett 8(2):328–331CrossRef
35.
Zurück zum Zitat Berend D, Tassa T (2010) Improved bounds on Bell numbers and on moments of sums of random variables. Probab Math Stat 30(2):185–205MathSciNetMATH Berend D, Tassa T (2010) Improved bounds on Bell numbers and on moments of sums of random variables. Probab Math Stat 30(2):185–205MathSciNetMATH
36.
Zurück zum Zitat Granmo O, Oommen BJ, Myrer SA, Olsen MG (2007) Learning automata-based solutions to the nonlinear fractional knapsack problem with applications to optimal resource allocation. IEEE Trans Syst Man Cybern B Cybern 37(1):166–175CrossRef Granmo O, Oommen BJ, Myrer SA, Olsen MG (2007) Learning automata-based solutions to the nonlinear fractional knapsack problem with applications to optimal resource allocation. IEEE Trans Syst Man Cybern B Cybern 37(1):166–175CrossRef
37.
Zurück zum Zitat Oommen BJ (1997) Stochastic searching on the line and its applications to parameter learning in nonlinear optimization. IEEE Trans Syst Man Cybern B Cybern 27(4):733–739MathSciNetCrossRef Oommen BJ (1997) Stochastic searching on the line and its applications to parameter learning in nonlinear optimization. IEEE Trans Syst Man Cybern B Cybern 27(4):733–739MathSciNetCrossRef
38.
Zurück zum Zitat Stege M, Jelitto J, Lohse N, Bronzel M, Fettweis G (1999) A stochastic vector channel model-implementation and verification. In: IEEE 50th Vehicular technology conference, vol 1, pp 97–101 Stege M, Jelitto J, Lohse N, Bronzel M, Fettweis G (1999) A stochastic vector channel model-implementation and verification. In: IEEE 50th Vehicular technology conference, vol 1, pp 97–101
39.
Zurück zum Zitat Liu Y, Elkashlan M, Ding Z, Karagiannidis GK (2016) Fairness of user clustering in MIMO non-orthogonal multiple access systems. IEEE Commun Lett 20(7):1465–1468 Liu Y, Elkashlan M, Ding Z, Karagiannidis GK (2016) Fairness of user clustering in MIMO non-orthogonal multiple access systems. IEEE Commun Lett 20(7):1465–1468
Metadaten
Titel
User grouping and power allocation in NOMA systems: a novel semi-supervised reinforcement learning-based solution
verfasst von
Rebekka Olsson Omslandseter
Lei Jiao
Yuanwei Liu
B. John Oommen
Publikationsdatum
14.07.2022
Verlag
Springer London
Erschienen in
Pattern Analysis and Applications / Ausgabe 1/2023
Print ISSN: 1433-7541
Elektronische ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-022-01091-2

Weitere Artikel der Ausgabe 1/2023

Pattern Analysis and Applications 1/2023 Zur Ausgabe

Premium Partner