Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Cover of the book

2018 | OriginalPaper | Chapter

Multiagent Learning Paradigms

Authors : K. Tuyls, P. Stone

Published in: Multi-Agent Systems and Agreement Technologies

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

“Perhaps a thing is simple if you can describe it fully in several different ways, without immediately knowing that you are describing the same thing” – Richard Feynman
This articles examines multiagent learning from several paradigmatic perspectives, aiming to bring them together within one framework. We aim to provide a general definition of multiagent learning and lay out the essential characteristics of the various paradigms in a systematic manner by dissecting multiagent learning into its main components. We show how these various paradigms are related and describe similar learning processes but from varying perspectives, e.g. an individual (cognitive) learner vs. a population of (simple) learning agents.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
next chapter Multiagent Resource Allocation: The Power and Limitations of Bilateral Deals (Extended Abstract)
Footnotes
1
Discounted utilities are used to represent that near-term payoffs are more important to the agent than longer term payoffs.
 
2
In some public auctions, the objective may instead be to maximize social welfare—striving to sell each item to the bidder who values it most.
 
Literature
1.
Albrecht, S.V., Stone, P.: Autonomous agents modelling other agents: a comprehensive survey and open problems. Artif. Intell. 258, 66–95 (2018)MathSciNetCrossRef
2.
Altshuler, Y., Bruckstein, A.M.: Static and expanding grid coverage with ant robots: complexity results. Theor. Comput. Sci. 412(35), 4661–4674 (2011)MathSciNetCrossRef
3.
Banerjee, A.: A simple model of herd behavior. Q. J. Econ. 107, 797–817 (1992)CrossRef
4.
Barrett, S., Stone, P., Kraus, S.: Empirical evaluation of ad hoc teamwork in the pursuit domain. In: 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2011), Taipei, Taiwan, 2–6 May, 2011, vol. 1–3, pp. 567–574 (2011)
5.
Bloembergen, D., Tuyls, K., Hennes, D., Kaisers, M.: Evolutionary dynamics of multi-agent learning: a survey. J. Artif. Intell. Res. 53, 659–697 (2015)MathSciNetCrossRef
6.
Broecker, B., Caliskanelli, I., Tuyls, K., Sklar, E.I., Hennes, D.: Hybrid insect-inspired multi-robot coverage in complex environments. In: Proceedings of the Towards Autonomous Robotic Systems - 16th Annual Conference, TAROS 2015, Liverpool, UK, 8–10 September 2015, pp. 56–68 (2015)CrossRef
7.
Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence and Tenth Innovative Applications of Artificial Intelligence Conference, AAAI 98, IAAI 98, Madison, Wisconsin, USA, 26–30 July, 1998, pp. 746–752 (1998)
8.
Colorni, A., Dorigo, M., Maniezzo, V.: Distributed optimization by ant colonies. In: Varela, F.J., Bourgine, P. (eds.) Towards a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, pp. 134–142. MIT Press, Cambridge (1992)
9.
Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)
10.
Fogel, D.B.: Evolving behaviors in the iterated prisoner’s dilemma. Evol. Comput. 1(1), 77–97 (1993)CrossRef
11.
Fogel, D.B.: Evolutionary computation - toward a new philosophy of machine intelligence. IEEE (1995)
12.
Galef, B.: Imitation in animals: history, definition, and interpretation of data from the psychological laboratory. In: Zentall, T., Galef, B. (eds.) Social Learning: Psychologicand Biological Perspectives. Lawrence Erlbaum Associates, Hillsdale (1988)
13.
Gatti, N., Restelli, M.: Sequence-form and evolutionary dynamics: realization equivalence to agent form and logit dynamics. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, USA, 12–17 February 2016, pp. 509–515 (2016)
14.
Genter, K.L., Stone, P.: Influencing a flock via ad hoc teamwork. In: Proceedings of the Swarm Intelligence - 9th International Conference, ANTS 2014, Brussels, Belgium, 10–12 September 2014, pp. 110–121 (2014)
15.
Gintis, H.: Game Theory Evolving, 2nd edn. University Press, Princeton (2009)MATH
16.
Hofbauer, J., Sigmund, K.: Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge (1998)
17.
Hu, J., Wellman, M.P.: Nash q-learning for general-sum stochastic games. J. Mach. Learn. Res. 4, 1039–1069 (2003)MathSciNetMATH
18.
Kaisers, M., Tuyls, K.: Frequency adjusted multi-agent q-learning. In: 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2010), Toronto, Canada, 10–14 May, 2010, vol. 1–3, pp. 309–316 (2010)
19.
20.
Knudson, M., Tumer, K.: Policy transfer in mobile robots using neuro-evolutionary navigation. In: Genetic and Evolutionary Computation Conference, GECCO 2012, Philadelphia, PA, USA, 7–11 July, 2012, Companion Material Proceedings, pp. 1411–1412 (2012)
21.
Laland, K., Richerson, P., Boyd, R.: Animal social learning: toward a new theoretical approach. In: Klopfer, P., Bateson, P., Thomson, N. (eds.) Perspectives in Ethology. Plenum Press, New York (1993)
22.
Lanctot, M.: Further developments of extensive-form replicator dynamics using the sequence-form representation. In: International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS 2014, Paris, France, 5–9 May, 2014, pp. 1257–1264 (2014)
23.
Littman, M.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the Eleventh International Conference on Machine Learning, pp. 157–163 (1994)
24.
Manderick, B., Spiessens, P.: Fine-grained parallel genetic algorithms. In: Proceedings of the 3rd International Conference on Genetic Algorithms, George Mason University, Fairfax, Virginia, USA, pp. 428–433, June 1989
25.
Maynard Smith, J., Price, G.R.: The logic of animal conflict. Nature 246(2), 15–18 (1973)CrossRef
26.
Mitchell, M.: An Introduction to Genetic Algorithms. MIT Press, Cambridge (1998)
27.
Mitchell, T.M.: Machine Learning. McGraw Hill Series in Computer Science. McGraw-Hill, New York (1997)MATH
28.
Palmer, G., Tuyls, K., Bloembergen, D., Savani, R.: Lenient multi-agent deep reinforcement learning. Accepted for AAMAS 2018 (2018)
29.
Panait, L., Tuyls, K., Luke, S.: Theoretical advantages of lenient learners: an evolutionary game theoretic perspective. J. Mach. Learn. Res. 9, 423–457 (2008)MathSciNetMATH
30.
Pardoe, D., Stone, P., Saar-Tsechansky, M., Keskin, T., Tomak, K.: Adaptive auction mechanism design and the incorporation of prior knowledge. INFORMS J. Comput. 22(3), 353–370 (2010)CrossRef
31.
Pardoe, D., Stone, P., Saar-Tsechansky, M., Tomak, K.: Adaptive mechanism design: a metalearning approach. In: Proceedings of the 8th International Conference on Electronic Commerce: The new e-commerce - Innovations for Conquering Current Barriers, Obstacles and Limitations to Conducting Successful Business on the Internet, 2006, Fredericton, New Brunswick, Canada, 13–16 August, 2006, pp. 92–102 (2006)
32.
Paredis, J.: Coevolutionary computation. Artif. Life 2(4), 355–375 (1995)CrossRef
33.
Parkes, D.C.: On Learnable Mechanism Design, p. 107–131. Springer-Verlag (2004)
34.
35.
Saravanan, N., Fogel, D.B.: Evolving neurocontrollers using evolutionary programming. In: Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence, Orlando, Florida, USA, 27–29 June, 1994, pp. 217–222 (1994)
36.
Shoham, Y., Leyton-Brown, K.: Multiagent Systems - Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge (2009)
37.
Shoham, Y., Powers, R., Grenager, T.: If multi-agent learning is the answer, what is the question? Artif. Intell. 171(7), 365–377 (2007)MathSciNetCrossRef
38.
39.
Stone, P., Veloso, M.M.: Multiagent systems: a survey from a machine learning perspective. Auton. Robots 8(3), 345–383 (2000)CrossRef
40.
Tuyls, K., Hoen, P.J., Vanschoenwinkel, B.: An evolutionary dynamical analysis of multi-agent learning in iterated games. Auton. Agents Multi-Agent Syst. 12(1), 115–153 (2006)CrossRef
41.
Tuyls, K., Parsons, S.: What evolutionary game theory tells us about multiagent learning. Artif. Intell. 171(7), 406–416 (2007)MathSciNetCrossRef
42.
Tuyls, K., Pérolat, J., Lanctot, M., Ostrovski, G., Savani, R., Leibo, J.Z., Ord, T., Graepel, T., Legg, S.: Symmetric decomposition of asymmetric games. Sci. Rep. 8(1), 1015 (2018)CrossRef
43.
Tuyls, K., Verbeeck, K., Lenaerts, T.: A selection-mutation model for q-learning in multi-agent systems. In: Proceedings of the Second International Joint Conference on Autonomous Agents & Multiagent Systems, AAMAS 2003, Melbourne, Victoria, Australia, 14–18 July, 2003, pp. 693–700 (2003)
44.
Tuyls, K., Weiss, G.: Multiagent learning: basics, challenges, and prospects. AI Mag. 33(3), 41–52 (2012)CrossRef
45.
Urzelai, J., Floreano, D.: Evolutionary robotics: coping with environment change. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2000), Las Vegas, Nevada, USA, 8–12 July, 2000, pp. 941–948 (2000)
46.
Weibull, J.W.: Evolutionary Game Theory. MIT Press, Cambridge (1997)
47.
Wooldridge, M.J.: Introduction to Multiagent Systems. Wiley, Hoboken (2002)
48.
Wunder, M., Littman, M.L., Babes, M.: Classes of multiagent q-learning dynamics with epsilon-greedy exploration. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, 21–24 June, 2010, pp. 1167–1174 (2010)
Metadata
Title
Multiagent Learning Paradigms
Authors
K. Tuyls
P. Stone
Copyright Year
2018
Book
Multi-Agent Systems and Agreement Technologies

Print ISBN: 978-3-030-01712-5

Electronic ISBN: 978-3-030-01713-2

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-030-01713-2_1

Premium Partner

    Image Credits