Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
The (Almost) Impossible Task of Interdisciplinarity Read chapter Read first chapter

2016 | OriginalPaper | Chapter

Anticipatory Behavior of Software Agents in Self-organizing Negotiations

Authors : Jan Ole Berndt, Otthein Herzog

Published in: Anticipation Across Disciplines

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Software agents are a well-established approach for modeling autonomous entities in distributed artificial intelligence. Iterated negotiations allow for coordinating the activities of multiple autonomous agents by means of repeated interactions. However, if several agents interact concurrently, the participants’ activities can mutually influence each other. This leads to poor coordination results. In this paper, we discuss these interrelations and propose a self-organization approach to cope with that problem. To that end, we apply distributed reinforcement learning as a feedback mechanism to the agents’ decision-making process. This enables the agents to use their experiences from previous activities to anticipate the results of potential future actions. They mutually adapt their behaviors to each other which results in the emergence of social order within the multiagent system. We empirically evaluate the dynamics of that process in a multiagent resource allocation scenario. The results show that the agents successfully anticipate the reactions to their activities in that dynamic and partially observable negotiation environment. This enables them to maximize their payoffs and to drastically outperform non-anticipating agents.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Information Concepts in Anticipatory Systems
next chapter The Ways of Scientific Anticipation: From Guesses to Probabilities and from There to Certainty
Footnotes
1
A famous example for this is the prisoner’s dilemma in which the equilibrium point is the only strategy combination not belonging to the Pareto frontier.
 
2
In this state of double contingency, both participants are unable to act because each of their activities depends on the other’s previous actions and they lack any existing expectations for selecting them. However, Luhmann notes that this is a highly unstable fixpoint of the interaction’s dynamics which never actually occurs in real encounters [15, 17]. Instead, every slight action allows for generating initial expectations which facilitate the emergence of social order.
 
3
All deviations are half-widths of the 99 % confidence interval.
 
Literature
1.
Berndt, J.O.: Self-organizing supply networks: autonomous agent coordination based on expectations. In: Filipe, J., Fred, A. (eds.) ICAART 2011, vol. 2, pp. 104–113. SciTePress, Rome (2011)
2.
Berndt, J.O.: Self-organizing logistics process control: an agent-based approach. In: Filipe, J., Fred, A. (eds.) Agents and Artificial Intelligence, pp. 397–412. Springer, Berlin (2013)CrossRef
3.
Berndt, J.O., Herzog, O.: Efficient multiagent coordination in dynamic environments. In: Boissier, O., Bradshaw, J., Cao, L., Fischer, K., Hacid, M.S. (eds.) WI-IAT 2011, pp. 188–195. IEEE Computer Society, Lyon (2011)
4.
Berndt, J.O., Herzog, O.: Distributed learning of best response behaviors in concurrent iterated many-object negotiations. In: Timm, I.J., Guttmann, C. (eds.) MATES 2012, pp. 15–29. Springer, Berlin (2012)
5.
Berndt, J.O., Herzog, O.: Distributed reinforcement learning for optimizing resource allocation in autonomous logistics processes. In: Kreowski, H.J., Scholz-Reiter, B., Thoben, K.D. (eds.) LDIC 2012, pp. 429–439. Springer, Berlin (2013)
6.
Buşoniu, L., Babuška, R., De Schutter, B.: Multi-agent reinforcement learning: an overview. In: Srinivasan, D., Jain, L. (eds.) Innovations in Multi-Agent Systems and Applications—1, pp. 183–221. Springer, Heidelberg (2010)
7.
Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: AAAI 1998. pp. 746–752. Madison, USA (1998)
8.
Cramton, P., Shoham, Y., Steinberg, R. (eds.): Combinatorial Auctions. The MIT Press, Cambridge (2006)MATH
9.
Endriss, U., Maudet, N., Sadri, F., Toni, F.: Negotiating socially optimal allocations of resources. J. Artif. Intell. Res. 25, 315–348 (2006)MathSciNet
10.
Faratin, P., Sierra, C., Jennings, N.R.: Negotiation decision functions for autonomous agents. Robot. Auton. Syst. 24(3–4), 159–182 (1998)CrossRef
11.
Foundation for Intelligent Physical Agents: FIPA Iterated Contract Net Interaction Protocol Specification. Standard (2002), document No. SC00030H
12.
13.
Jennings, N.R., Faratin, P., Lomuscio, A.R., Parsons, S., Wooldridge, M.J., Sierra, C.: Automated negotiation: prospects. Methods Chall. Group Decis. Negoti. 10, 199–215 (2001)CrossRef
14.
Luckhart, C., Irani, K.B.: An algorithmic solution of N-person games. In: AAAI 1986. vol. 1, pp. 158–162. Morgan Kaufmann, Philadelphia, USA (1986)
15.
Luhmann, N.: Soziale Systeme. Grundriß einer allgemeinen Theorie. Suhrkamp, Frankfurt (1984)
16.
Luhmann, N.: Probleme mit operativer Schließung. In: Luhmann, N. (ed.) Die Soziologie und der Mensch, Soziologische Aufklärung, vol. 6, pp. 12–24. Westdeutscher Verlag, Opladen (1995)
17.
Luhmann, N.: Social Systems. Stanford University Press, Stanford (1995)
18.
Mazur, D.R.: Combinatorics. A guided tour. MAA Textbooks, The Mathematical Association of America, Washington (2010)MATH
19.
20.
Porter, R., Nudelman, E., Shoham, Y.: Simple search methods for finding a Nash equilibrium. Game. Econ. Behav. 63(2), 642–662 (2008)MATHCrossRefMathSciNet
21.
Ramezani, S., Endriss, U.: Nash social welfare in multiagent resource allocation. In: David, E., Gerding, E., Sarne, D., Shehory, O. (eds.) Agent-Mediated Electronic Commerce, pp. 117–131. Springer, Heidelberg (2010)
22.
Schuldt, A.: Multiagent coordination enabling autonomous logistics. Springer, Heidelberg (2011)MATHCrossRef
23.
Schuldt, A., Berndt, J.O., Herzog, O.: The interaction effort in autonomous logistics processes: potential and limitations for cooperation. In: Hülsmann, M., Scholz-Reiter, B., Windt, K. (eds.) Autonomous Cooperation and Control in Logistics, pp. 77–90. Springer, Berlin (2011)CrossRef
24.
Schuldt, A., Gehrke, J.D., Werner, S.: Designing a simulation middleware for FIPA multiagent systems. In: Jain, L., Gini, M., Faltings, B.B., Terano, T., Zhang, C., Cercone, N., Cao, L. (eds.) WI-IAT 2008, pp. 109–113. IEEE Computer Society Press, Sydney (2008)
25.
Schuldt, A., Werner, S.: Distributed Clustering of Autonomous Shipping Containers by Concept, Location, and Time. In: Müller, J.P., Petta, P., Klusch, M., Georgeff, M. (eds.) MATES 2007, pp. 121–132. Springer, Berlin (2007)
26.
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (1998)
27.
van Bragt, D.D.B., La Poutré, J.A.: Why Agents for Automated Negotiations Should Be Adaptive. Netnomics 5(2), 101–118 (2003)CrossRef
28.
29.
von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944)MATH
30.
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)MATH
31.
Wooldridge, M., Jennings, N.R.: Intelligent agents: theory and practice. Knowl. Eng. Rev. 10(2), 115–152 (1995)CrossRef
32.
Metadata
Title
Anticipatory Behavior of Software Agents in Self-organizing Negotiations
Authors
Jan Ole Berndt
Otthein Herzog
Copyright Year
2016
Book
Anticipation Across Disciplines

Print ISBN: 978-3-319-22598-2

Electronic ISBN: 978-3-319-22599-9

Copyright Year: 2016

DOI
https://doi.org/10.1007/978-3-319-22599-9_15

Premium Partner

    Image Credits