Skip to main content

2020 | OriginalPaper | Buchkapitel

Q-routing: From the Algorithm to the Routing Protocol

verfasst von : Alexis Bitaillou, Benoît Parrein, Guillaume Andrieux

Erschienen in: Machine Learning for Networking

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Routing is a complex task in computer network. This function is mainly devoted to the layer 3 in the Open Standard Interconnection (OSI) model. In the 90s, routing protocols assisted by reinforcement learning were created. To illustrate the performance, most of the literature use centralized algorithms and “home-made” simulators that make difficult (i) the transposition to real networks; (ii) the reproducibility. The goal of this work is to address those 2 points. In this paper, we propose a complete distributed protocol implementation. We deployed the routing algorithm proposed by Boyan and Littman in 1994 based on Q-learning on the network simulator Qualnet. Twenty-five years later, we conclude that a more realistic implementation in more realistic network environment does not give always better Quality of Service than the historical Bellman-Ford protocol. We provide all the materials to conduct reproducible research.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
Literatur
1.
Zurück zum Zitat Arroyo-Valles, R., Alaiz-Rodriguez, R., Guerrero-Curieses, A., Cid-Sueiro, J.: Q-probabilistic routing in wireless sensor networks. In: Sensor Networks and Information 2007 3rd International Conference on Intelligent Sensors, pp. 1–6, December 2007. https://doi.org/10.1109/ISSNIP.2007.4496810 Arroyo-Valles, R., Alaiz-Rodriguez, R., Guerrero-Curieses, A., Cid-Sueiro, J.: Q-probabilistic routing in wireless sensor networks. In: Sensor Networks and Information 2007 3rd International Conference on Intelligent Sensors, pp. 1–6, December 2007. https://​doi.​org/​10.​1109/​ISSNIP.​2007.​4496810
3.
Zurück zum Zitat Boyan, J.A., Littman, M.L.: Packet routing in dynamically changing networks: a reinforcement learning approach. In: Advances in Neural Information Processing Systems, pp. 671–678 (1994) Boyan, J.A., Littman, M.L.: Packet routing in dynamically changing networks: a reinforcement learning approach. In: Advances in Neural Information Processing Systems, pp. 671–678 (1994)
4.
Zurück zum Zitat Choi, S.P., Yeung, D.Y.: Predictive Q-routing: a memory-based reinforcement learning approach to adaptive traffic control. In: Advances in Neural Information Processing Systems, pp. 945–951 (1996) Choi, S.P., Yeung, D.Y.: Predictive Q-routing: a memory-based reinforcement learning approach to adaptive traffic control. In: Advances in Neural Information Processing Systems, pp. 945–951 (1996)
6.
Zurück zum Zitat Gupta, N., et al.: Improved route selection approaches using Q-learning framework for 2D NoCs. In: Proceedings of the 3rd International Workshop on Many-core Embedded Systems, MES 2015, Portland, OR, USA, pp. 33–40. ACM, New York (2015). https://doi.org/10.1145/2768177.2768180 Gupta, N., et al.: Improved route selection approaches using Q-learning framework for 2D NoCs. In: Proceedings of the 3rd International Workshop on Many-core Embedded Systems, MES 2015, Portland, OR, USA, pp. 33–40. ACM, New York (2015). https://​doi.​org/​10.​1145/​2768177.​2768180
8.
Zurück zum Zitat Kumar, S.: Dual reinforcement Q-routing: an on-line adaptive routing algorithm. In: 1997 Proceedings of the Artificial Neural Networks in Engineering Conference (1997) Kumar, S.: Dual reinforcement Q-routing: an on-line adaptive routing algorithm. In: 1997 Proceedings of the Artificial Neural Networks in Engineering Conference (1997)
10.
11.
Zurück zum Zitat Schooenderwoerd, R., Holland, O., Bruten, J., Rosenkrantz, L.: Ants for load balancing in telecommunication networks. Technical report 96-35, HP Labs, Bristol (1996) Schooenderwoerd, R., Holland, O., Bruten, J., Rosenkrantz, L.: Ants for load balancing in telecommunication networks. Technical report 96-35, HP Labs, Bristol (1996)
13.
Zurück zum Zitat Xia, B., Wahab, M.H., Yang, Y., Fan, Z., Sooriyabandara, M.: Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks. In: 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications, pp. 1–5, June 2009. https://doi.org/10.1109/CROWNCOM.2009.5189189 Xia, B., Wahab, M.H., Yang, Y., Fan, Z., Sooriyabandara, M.: Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks. In: 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications, pp. 1–5, June 2009. https://​doi.​org/​10.​1109/​CROWNCOM.​2009.​5189189
14.
Zurück zum Zitat Yap, S.T., Othman, M.: An adaptive routing algorithm: enhanced confidence-based Q routing algorithm in network traffic. Malays. J. Comput. Sci. 17(2), 21–29 (2004) Yap, S.T., Othman, M.: An adaptive routing algorithm: enhanced confidence-based Q routing algorithm in network traffic. Malays. J. Comput. Sci. 17(2), 21–29 (2004)
Metadaten
Titel
Q-routing: From the Algorithm to the Routing Protocol
verfasst von
Alexis Bitaillou
Benoît Parrein
Guillaume Andrieux
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-45778-5_5