nach oben

Erschienen in:

2021 | OriginalPaper | Buchkapitel

Governing Black-Box Agents in Competitive Multi-Agent Systems

verfasst von : Michael Pernpeintner, Christian Bartelt, Heiner Stuckenschmidt

Erschienen in: Multi-Agent Systems

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Competitive Multi-Agent Systems (MAS) are inherently hard to control due to agent autonomy and strategic behavior, which is particularly problematic when there are system-level objectives to be achieved or specific environmental states to be avoided.

Existing solutions for this task mostly assume specific knowledge about agent preferences, utilities and strategies, neglecting the fact that actions are not always directly linked to genuine agent preferences, but can also reflect anticipated competitor behavior, be a concession to a superior adversary or simply be intended to mislead other agents. This assumption both reduces applicability to real-world systems and opens room for manipulation.

We therefore propose a new governance approach for competitive MAS which relies exclusively on publicly observable actions and transitions, and uses the acquired knowledge to purposefully restrict action spaces, thereby achieving the system’s objectives while preserving a high level of autonomy for the agents.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Ascending-Price Mechanism for General Multi-sided Markets

Nächstes Kapitel Path and Action Planning in Non-uniform Environments for Multi-agent Pickup and Delivery Tasks

Andrighetto, G., Governatori, G., Noriega, P., van der Torre, L.: Normative multi-agent systems (2013). https://doi.org/10.4230/DFU.Vol4.12111.i

Asadi, M., Huber, M.: State space reduction for hierarchical reinforcement learning, January 2004

Bade, S.: Nash equilibrium in games with incomplete preferences. Econ. Theory 26(2), 309–332 (2005). www.jstor.org/stable/25055952

Balke, T., et al.: Norms in MAS: definitions and related concepts, p. 31, January 2013

Boella, G., van der Torre, L., Verhagen, H.: Introduction to normative multiagent systems. Comput. Math. Organ. Theory 12(2), 71–79 (2006). https://doi.org/10.1007/s10588-006-9537-7CrossRef

Brafman, R.I., Tennenholtz, M.: On partially controlled multi-agent systems. J. Artif. Int. Res. 4(1), 477–507 (1996)MathSciNetMATH

Bulling, N., Dastani, M.: Norm-based mechanism design. Artif. Intell. 239(C), 97–142 (2016). https://doi.org/10.1016/j.artint.2016.07.001MathSciNetCrossRefMATH

Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence, AAAI 1998/IAAI 1998, pp. 746–752. American Association for Artificial Intelligence, Menlo Park (1998). http://dl.acm.org/citation.cfm?id=295240.295800

Dean, T., Givan, R., Leach, S.: Model reduction techniques for computing approximately optimal solutions for Markov decision processes. In: Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, pp. 124–131. UAI 1997, Morgan Kaufmann Publishers Inc., San Francisco (1997)

10.

Dell’Anna, D., Dastani, M., Dalpiaz, F.: Runtime revision of norms and sanctions based on agent preferences. In: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2019, pp. 1609–1617. International Foundation for Autonomous Agents and Multiagent Systems, Richland (2019). Event-place: Montreal QC, Canada

11.

Durugkar, I., Liebman, E., Stone, P.: Balancing individual preferences and shared objectives in multiagent reinforcement learning, p. 2483, July 2020. https://doi.org/10.24963/ijcai.2020/343

12.

Fitoussi, D., Tennenholtz, M.: Choosing social laws for multi-agent systems: minimality and simplicity. Artif. Intell. 119(1), 61–101 (2000)MathSciNetCrossRef

13.

García-Camino, A., Rodríguez-Aguilar, J., Sierra, C., Vasconcelos, W.: A rule-based approach to norm-oriented programming of electronic institutions. SIGecom Exchanges 5, 33–40 (2006)

14.

Gutierrez, J., Perelli, G., Wooldridge, M.: Imperfect information in reactive modules games. Inf. Comput. 261, 650–675 (2018)MathSciNetCrossRef

15.

Hernandez-Leal, P., Kartal, B., Taylor, M.E.: A survey and critique of multiagent deep reinforcement learning. Auton. Agent. Multi-Agent Syst. 33(6), 750–797 (2019). https://doi.org/10.1007/s10458-019-09421-1CrossRef

16.

Hoen, P.J., Tuyls, K., Panait, L., Luke, S., La Poutré, J.A.: An overview of cooperative and competitive multiagent learning. In: Tuyls, K., Hoen, P.J., Verbeeck, K., Sen, S. (eds.) LAMAS 2005. LNCS (LNAI), vol. 3898, pp. 1–46. Springer, Heidelberg (2006). https://doi.org/10.1007/11691839_1CrossRef

17.

Kim, S., Lewis, M.E., White, C.C.: State space reduction for nonstationary stochastic shortest path problems with real-time traffic information. IEEE Trans. Intell. Transp. Syst. 6(3), 273–284 (2005). https://doi.org/10.1109/TITS.2005.853695CrossRef

18.

Koriche, F., Zanuttini, B.: Learning conditional preference networks. Artif. Intell. 174(11), 685–703 (2010). https://doi.org/10.1016/j.artint.2010.04.019. http://www.sciencedirect.com/science/article/pii/S000437021000055X

19.

Lecarpentier, E., Rachelson, E.: Non-stationary Markov decision processes a worst-case approach using model-based reinforcement learning, April 2019

20.

Levy, Y.J., Solan, E.: Stochastic games. In: Meyers, R.A. (ed.) Encyclopedia of Complexity and Systems Science, pp. 1–23. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-642-27737-5_522-2CrossRef

21.

Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the eleventh international conference on international conference on machine learning, ICML1994, pp. 157–163. Morgan Kaufmann Publishers Inc., San Francisco (1994)

22.

Liu, L., Chattopadhyay, A., Mitra, U.: On solving MDPs with large state space: exploitation of policy structures and spectral properties. IEEE Trans. Commun. 67(6), 4151–4165 (2019). https://doi.org/10.1109/TCOMM.2019.2899620CrossRef

23.

Liu, T., Wang, J., Zhang, X., Cheng, D.: Game theoretic control of multiagent systems. SIAM J. Control. Optim. 57, 1691–1709 (2019)MathSciNetCrossRef

24.

Lüdtke, S., Schröder, M., Krüger, F., Bader, S., Kirste, T.: State-space abstractions for probabilistic inference: a systematic review. J. Artif. Int. Res. 63(1), 789–848 (2018). https://doi.org/10.1613/jair.1.11261MathSciNetCrossRefMATH

25.

Majeed, S.J., Hutter, M.: Exact reduction of huge action spaces in general reinforcement learning (2020)

26.

Meyer, J.J.C., Wieringa, R.J. (eds.): Deontic Logic in Computer Science: Normative System Specification. Wiley, USA (1994)

27.

Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning,ICML2016, vol. 48. pp. 1928–1937. JMLR.org (2016)

28.

Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–33 (2015)CrossRef

29.

Morales, J.: On-line norm synthesis for open multi-agent systems. Ph.D. thesis, Universitat de Barcelona (2016)

30.

Morris-Martin, A., De Vos, M., Padget, J.: Norm emergence in multiagent systems: a viewpoint paper. Auton. Agents Multi-Agent Syst. 33(6), 706–749 (2019). https://doi.org/10.1007/s10458-019-09422-0CrossRef

31.

Nowé, A., Vrancx, P., De Hauwere, Y.M.: Game theory and multi-agent reinforcement learning. In: Wiering, M., van Otterlo, M. (eds.) Reinforcement Learning: State-of-the-Art, pp. 441–470. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27645-3_14CrossRef

32.

Perelli, G.: Enforcing equilibria in multi-agent systems. In: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2019, pp. 188–196. International Foundation for Autonomous Agents and Multiagent Systems, Richland (2019). Event-place: Montreal QC, Canada

33.

Relund Nielsen, L., Jørgensen, E., Højsgaard, S.: Embedding a state space model into a Markov decision process. Ann. Oper. Res. 190(1), 289–309 (2011). https://doi.org/10.1007/s10479-010-0688-zMathSciNetCrossRefMATH

34.

Rotolo, A.: Norm compliance of rule-based cognitive agents. In: IJCAI International Joint Conference on Artificial Intelligence, pp. 2716–2721, January 2011

35.

Rotolo, A., van der Torre, L.: Rules, agents and norms: guidelines for rule-based normative multi-agent systems. In: Bassiliades, N., Governatori, G., Paschke, A. (eds.) RuleML 2011. LNCS, vol. 6826, pp. 52–66. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22546-8_6CrossRef

36.

Shapley, L.S.: Stochastic games. Proc. Natl. Acad. Sci. U.S.A. 39(10), 1095–1100 (1953). https://pubmed.ncbi.nlm.nih.gov/16589380

37.

Shoham, Y., Powers, R., Grenager, T.: Multi-agent reinforcement learning: a critical survey, June 2003

38.

Shoham, Y., Tennenholtz, M.: On social laws for artificial agent societies: off-line design. Artif. Intell. 73(1), 231–252 (1995). https://doi.org/10.1016/0004-3702(94)00007-N. http://www.sciencedirect.com/science/article/pii/000437029400007N

39.

Stirling, W.C., Felin, T.: Game theory, conditional preferences, and social influence. PLOS One 8(2), 1–11 (2013). https://doi.org/10.1371/journal.pone.0056751CrossRef

40.

Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. A Bradford Book, Cambridge (2018)MATH

41.

Watkins, C.: Learning from delayed rewards, January 1989

42.

Wolf, T.D., Holvoet, T.: Emergence and self-organisation: a statement of similarities and differences (2004)

Titel: Governing Black-Box Agents in Competitive Multi-Agent Systems
verfasst von: Michael Pernpeintner
Christian Bartelt
Heiner Stuckenschmidt
Verlag: Springer International Publishing
Buch: Multi-Agent Systems
Print ISBN: 978-3-030-82253-8

Electronic ISBN: 978-3-030-82254-5

Copyright-Jahr: 2021
DOI: https://doi.org/10.1007/978-3-030-82254-5_2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"