Skip to main content

2021 | OriginalPaper | Buchkapitel

Convergence of the Reinforcement Learning Mechanism Applied to the Channel Detection Sequence Problem

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The use of mechanisms based on artificial intelligence techniques to perform dynamic learning has received much attention recently and has been applied in solving many problems. However, the convergence analysis of these mechanisms does not always receive the same attention. In this paper, the convergence of the mechanism using reinforcement learning to determine the channel detection sequence in a multi-channel, multi-user radio network is discussed and, through simulations, recommendations are presented for the proper choice of the learning parameter set to improve the overall reward. Then, applying the related set of parameters to the problem, the mechanism is compared to other intuitive sorting mechanisms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat McHenry, M.A.: NSF Spectrum Occupancy Measurements Project (2005) McHenry, M.A.: NSF Spectrum Occupancy Measurements Project (2005)
2.
Zurück zum Zitat FCC: FCC-03-322 - NOTICE OF PROPOSED RULE MAKING AND ORDER. Technical report, Federal Communications Commission, 30 December 2003 FCC: FCC-03-322 - NOTICE OF PROPOSED RULE MAKING AND ORDER. Technical report, Federal Communications Commission, 30 December 2003
3.
Zurück zum Zitat Cheng, H.T., Zhuang, W.: Simple channel sensing order in cognitive radio networks. IEEE J. Sel. Areas Commun. (2011) Cheng, H.T., Zhuang, W.: Simple channel sensing order in cognitive radio networks. IEEE J. Sel. Areas Commun. (2011)
4.
Zurück zum Zitat Chow, Y.S., Robbins, H., Siegmund, D.: Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin Company, Boston (1971) Chow, Y.S., Robbins, H., Siegmund, D.: Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin Company, Boston (1971)
5.
Zurück zum Zitat Mendes, A.C., Augusto, C.H.P., Da Silva, M.W., Guedes, R.M., De Rezende, J.F.: Channel sensing order for cognitive radio networks using reinforcement learning. In: IEEE LCN (2011) Mendes, A.C., Augusto, C.H.P., Da Silva, M.W., Guedes, R.M., De Rezende, J.F.: Channel sensing order for cognitive radio networks using reinforcement learning. In: IEEE LCN (2011)
6.
Zurück zum Zitat Claus, C., Boutilier, C.: The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems. National Conference on Artificial Intelligence (1998) Claus, C., Boutilier, C.: The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems. National Conference on Artificial Intelligence (1998)
7.
Zurück zum Zitat Tan, M.: Multi-agent Reinforcement Learning: Independent vs. Cooperative Agents. In: Readings in Agents (1997) Tan, M.: Multi-agent Reinforcement Learning: Independent vs. Cooperative Agents. In: Readings in Agents (1997)
8.
Zurück zum Zitat Lauer, M., Riedmiller, M.: An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In: ICML (2000) Lauer, M., Riedmiller, M.: An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In: ICML (2000)
9.
Zurück zum Zitat Kapetanakis, S., Kudenko, D.: Improving on the reinforcement learning of coordination in cooperative multi-agent systems. In: AAMAS (2002) Kapetanakis, S., Kudenko, D.: Improving on the reinforcement learning of coordination in cooperative multi-agent systems. In: AAMAS (2002)
10.
Zurück zum Zitat Lauer, M., Riedmiller, M.: Reinforcement learning for stochastic cooperative multiagent systems. In: AAMAS (2004) Lauer, M., Riedmiller, M.: Reinforcement learning for stochastic cooperative multiagent systems. In: AAMAS (2004)
11.
Zurück zum Zitat Bowling, M.: Convergence and No-Regret in Multiagent Learning. In: Advances in Neural Information Processing Systems 17. MIT Press, Cambridge (2005) Bowling, M.: Convergence and No-Regret in Multiagent Learning. In: Advances in Neural Information Processing Systems 17. MIT Press, Cambridge (2005)
12.
Zurück zum Zitat Jafari, A., Greenwald, A., Gondek, D., Ercal, G.: On no-regret learning, fictitious play and nash equilibrium. In: Proceedings of the 18th International Conference on Machine Learning (2001) Jafari, A., Greenwald, A., Gondek, D., Ercal, G.: On no-regret learning, fictitious play and nash equilibrium. In: Proceedings of the 18th International Conference on Machine Learning (2001)
13.
Zurück zum Zitat Zapechelnyuk, A.: Limit behavior of no-regret dynamics. Technical report, School of Economics, Kyiv, Ucraine (2009) Zapechelnyuk, A.: Limit behavior of no-regret dynamics. Technical report, School of Economics, Kyiv, Ucraine (2009)
14.
Zurück zum Zitat Leslie, D., Collins, E.: Generalised weakened fctitious play. Games Econ. Behav. 56(2) (2006) Leslie, D., Collins, E.: Generalised weakened fctitious play. Games Econ. Behav. 56(2) (2006)
15.
Zurück zum Zitat Brown, G.: Some notes on computation of games solutions. Research memoranda rm-125-pr, RAND Corporation, Santa Monica, California (1949) Brown, G.: Some notes on computation of games solutions. Research memoranda rm-125-pr, RAND Corporation, Santa Monica, California (1949)
16.
Zurück zum Zitat Verbeeck, K., Nowé, A., Parent, J., Tuyls, K.: Exploring selfish reinforcement learning in repeated games with stochastic rewards. In: JAAMAS (2006) Verbeeck, K., Nowé, A., Parent, J., Tuyls, K.: Exploring selfish reinforcement learning in repeated games with stochastic rewards. In: JAAMAS (2006)
17.
Zurück zum Zitat Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MP, Cambridge (1998) Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MP, Cambridge (1998)
18.
Zurück zum Zitat Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)MATH Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)MATH
19.
Zurück zum Zitat Yau, K.A., Komisarczuk, P., Teal, P.D.: Applications of reinforcement learning to cognitive radio networks. In: IEEE International Conference in Communications (ICC) (July 2010) Yau, K.A., Komisarczuk, P., Teal, P.D.: Applications of reinforcement learning to cognitive radio networks. In: IEEE International Conference in Communications (ICC) (July 2010)
20.
Zurück zum Zitat Yau, K.A., Komisarczuk, P., Teal, P.D.: Enhancing network performance in distributed cognitive radio networks using single-agent and multi-agent reinforcement learning. In: IEEE Conference on Local Computer Networks (October 2010) Yau, K.A., Komisarczuk, P., Teal, P.D.: Enhancing network performance in distributed cognitive radio networks using single-agent and multi-agent reinforcement learning. In: IEEE Conference on Local Computer Networks (October 2010)
21.
Zurück zum Zitat Vu, H.L., Sakurai, T.: Collision probability in saturated IEEE 802.11 networks. In: Australian Telecommunication Networks and Applications Conference (2006) Vu, H.L., Sakurai, T.: Collision probability in saturated IEEE 802.11 networks. In: Australian Telecommunication Networks and Applications Conference (2006)
22.
Zurück zum Zitat Hasselt, H.: Double q-learning. In: NIPS (2010) Hasselt, H.: Double q-learning. In: NIPS (2010)
Metadaten
Titel
Convergence of the Reinforcement Learning Mechanism Applied to the Channel Detection Sequence Problem
verfasst von
André Mendes
Copyright-Jahr
2021
DOI
https://doi.org/10.1007/978-3-030-91885-9_30

Premium Partner