Skip to main content

2018 | OriginalPaper | Buchkapitel

Accelerating the Computation of Solutions in Resource Allocation Problems Using an Evolutionary Approach and Multiagent Reinforcement Learning

verfasst von : Ana L. C. Bazzan

Erschienen in: Applications of Evolutionary Computation

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In systems composed by a high number of highly coupled components, aligning the optimum of the system with the optimum of those individual components can be conflicting, especially in situations in which resources are scarce. In order to deal with this, many authors have proposed forms of biasing the optimization process. However, mostly, this works for cooperative scenarios. When resources are scarce, the components compete for them, thus those solutions are not necessarily appropriate. In this paper a new approach is proposed, in which there is a synergy between: (i) a global optimization process in which the system authority employs metaheuristics, and (ii) reinforcement learning processes that run at each component or agent. Both the agents and the system authority exchange solutions that are incorporated by the other party. The contributions are twofold: we propose a general scheme for such synergy and show its benefits in scenarios related to congestion games.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
A learning episode for the RL coincides with a generation for the metaheuristic.
 
Literatur
1.
Zurück zum Zitat Bazzan, A.L.C., Chira, C.: Hybrid evolutionary and reinforcement learning approach to accelerate traffic assignment (extended abstract). In: Bordini, R., Elkind, E., Weiss, G., Yolum, P. (eds.) IFAAMAS Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), pp. 1723–1724, May 2015. http://www.aamas2015.com/en/AAMAS_2015_USB/aamas/p1723.pdf Bazzan, A.L.C., Chira, C.: Hybrid evolutionary and reinforcement learning approach to accelerate traffic assignment (extended abstract). In: Bordini, R., Elkind, E., Weiss, G., Yolum, P. (eds.) IFAAMAS Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), pp. 1723–1724, May 2015. http://​www.​aamas2015.​com/​en/​AAMAS_​2015_​USB/​aamas/​p1723.​pdf
2.
Zurück zum Zitat Chira, C., Bazzan, A.L.C., Rossetti, R.J.F.: Multi-objective evolutionary traffic assignment. In: Proceedings of the 18th IEEE Annual Conference on Intelligent Transport Systems (ITSC 2015), pp. 1177–1182. IEEE, September 2015. https://doi.org/10.1109/ITSC.2015.194 Chira, C., Bazzan, A.L.C., Rossetti, R.J.F.: Multi-objective evolutionary traffic assignment. In: Proceedings of the 18th IEEE Annual Conference on Intelligent Transport Systems (ITSC 2015), pp. 1177–1182. IEEE, September 2015. https://​doi.​org/​10.​1109/​ITSC.​2015.​194
3.
Zurück zum Zitat Colby, M., Tumer, K.: Shaping fitness functions for coevolving cooperative multiagent systems. In: Proceedings of the Eleventh International Joint Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain, pp. 425–432, June 2012 Colby, M., Tumer, K.: Shaping fitness functions for coevolving cooperative multiagent systems. In: Proceedings of the Eleventh International Joint Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain, pp. 425–432, June 2012
4.
Zurück zum Zitat D’Acierno, L., Montella, B., De Lucia, F.: A stochastic traffic assignment algorithm based on ant colony optimisation. In: Dorigo, M., Gambardella, L.M., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.) ANTS 2006. LNCS, vol. 4150, pp. 25–36. Springer, Heidelberg (2006). https://doi.org/10.1007/11839088_3 CrossRef D’Acierno, L., Montella, B., De Lucia, F.: A stochastic traffic assignment algorithm based on ant colony optimisation. In: Dorigo, M., Gambardella, L.M., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.) ANTS 2006. LNCS, vol. 4150, pp. 25–36. Springer, Heidelberg (2006). https://​doi.​org/​10.​1007/​11839088_​3 CrossRef
5.
Zurück zum Zitat Hines, G., Larson, K.: Learning when to take advice: a statistical test for achieving a correlated equilibrium. In: McAllester, D.A., Myllymki, P. (eds.) UAI, pp. 274–281. AUAI Press (2008) Hines, G., Larson, K.: Learning when to take advice: a statistical test for achieving a correlated equilibrium. In: McAllester, D.A., Myllymki, P. (eds.) UAI, pp. 274–281. AUAI Press (2008)
6.
Zurück zum Zitat LeBlanc, L.J., Morlok, E.K., Pierskalla, W.P.: An efficient approach to solving the road network equilibrium traffic assignment problem. Transp. Res. 9(5), 309–318 (1975)CrossRef LeBlanc, L.J., Morlok, E.K., Pierskalla, W.P.: An efficient approach to solving the road network equilibrium traffic assignment problem. Transp. Res. 9(5), 309–318 (1975)CrossRef
7.
Zurück zum Zitat de Lima Jr., F., Neto, A., de Melo, J.: Hybrid Metaheuristics Using Reinforcement Learning Applied to Salesman Traveling Problem. INTECH Open Access Publisher (2010) de Lima Jr., F., Neto, A., de Melo, J.: Hybrid Metaheuristics Using Reinforcement Learning Applied to Salesman Traveling Problem. INTECH Open Access Publisher (2010)
8.
Zurück zum Zitat Stefanello, F., Buriol, L., Hirsch, M., Pardalos, P., Querido, T., Resende, M., Ritt, M.: On the minimization of traffic congestion in road networks with tolls. Ann. Oper. Res. 249, 1–21 (2015)MathSciNetMATH Stefanello, F., Buriol, L., Hirsch, M., Pardalos, P., Querido, T., Resende, M., Ritt, M.: On the minimization of traffic congestion in road networks with tolls. Ann. Oper. Res. 249, 1–21 (2015)MathSciNetMATH
9.
Zurück zum Zitat Wardrop, J.G.: Some theoretical aspects of road traffic research. Proc. Inst. Civ. Eng. Part II 1(36), 325–362 (1952) Wardrop, J.G.: Some theoretical aspects of road traffic research. Proc. Inst. Civ. Eng. Part II 1(36), 325–362 (1952)
10.
Zurück zum Zitat Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)MATH Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)MATH
11.
Zurück zum Zitat Wolpert, D.H., Sill, J., Tumer, K.: Reinforcement learning in distributed domains: beyond team games. In: International Joint Conferences on Artificial Intelligence (IJCAI), pp. 819–824 (2001) Wolpert, D.H., Sill, J., Tumer, K.: Reinforcement learning in distributed domains: beyond team games. In: International Joint Conferences on Artificial Intelligence (IJCAI), pp. 819–824 (2001)
13.
Zurück zum Zitat Zhang, C., Abdallah, S., Lesser, V.: Integrating organizational control into multi-agent learning. In: ao Sichman, J.S., Decker, K.S., Sierra, C., Castelfranchi, C. (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Budapest, Hungary, pp. 757–764 (2009). http://mas.cs.umass.edu/paper/465 Zhang, C., Abdallah, S., Lesser, V.: Integrating organizational control into multi-agent learning. In: ao Sichman, J.S., Decker, K.S., Sierra, C., Castelfranchi, C. (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Budapest, Hungary, pp. 757–764 (2009). http://​mas.​cs.​umass.​edu/​paper/​465
Metadaten
Titel
Accelerating the Computation of Solutions in Resource Allocation Problems Using an Evolutionary Approach and Multiagent Reinforcement Learning
verfasst von
Ana L. C. Bazzan
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-77538-8_14

Premium Partner