nach oben

Journal of Economic Interaction and Coordination

Erschienen in:

09.04.2017 | Regular Article

The economics analysis of a Q-learning model of cooperation with punishment and risk taking preferences

verfasst von: Nazaria Solferino, Viviana Solferino, Serena F. Taurino

Erschienen in: Journal of Economic Interaction and Coordination | Ausgabe 3/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The aim of this paper is to better understand how cooperation mechanisms work in the context of a Q-learning model. We apply a learning reinforcement model to analyse the conditions needed to have a stable cooperative equilibrium when people take part in a common project and could take advantages of free-riding. Our results show that a stable equilibrium can be reached thank to mechanisms of punishment, but the final result strongly depends on the risk-taking individuals’ preferences. In particular, we find that the penalties will be effective only with people having high exploration rates,namely with people able to adapt their strategies and learn to cooperate. Otherwise, it is possible to have an unstable equilibrium with cooperation until individuals have a very high intrinsic motivation to cooperate, whatever the others do.

Vorheriger Artikel The creative response and the endogenous dynamics of pecuniary knowledge externalities: an agent based simulation model

Nächster Artikel Zero-intelligence agents looking for a job

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

In this work for the sake of simplicity we do not consider a forward looking analysis as well. In fact for the purpose of the work we are just interested in detecting players’ strategies when the agents can free-ride, aware that they may earn a sufficiently consistent future reward from the production of that common good.

For some theory literature that exists on the convergence to or not to Nash equilibrium in reinforcement learning models and exploration-based model see for example Beggs (2005) and Young (2009).

Andreoni J (1989) Giving with impure altruism: applications to charities and Ricardian equivalence. J Polit Econ 97(6):1447–1458

Andreoni J (1990) Impure altruism and donations to public goods: a theory of warm glow giving. Econ J 100(401):464–477CrossRef

Becchetti L, Pelligra V, Taurino SF (2015) Other regarding preferences and reciprocity: insights from experimental findings and satisfaction. Data CEIS working paper no. 363

Beggs AW (2005) On the convergence of reinforcement learning. J Econ Theory 122:1–36CrossRef

Boyd R, Richerson PJ (1988) The evolution of reciprocity in sizeable groups. J Theor Biol 131:337–356CrossRef

Boyd R, Richerson PJ (1992) Punishment allows the evolution of cooperation (or anything else) in sizable groups. Ethol Sociobiol 13(5):171–195CrossRef

Brandt H, Hauert C, Sigmund K (2006) Punishing and abstaining for public goods. Proc Natl Acad Sci 103:495–497CrossRef

Bruni L (2006) Reciprocità. Dinamiche di cooperazione, economia e società civile. Mondadori Eds, Milan

Burton-Chellew MN, Nax HH, West SA (2015) Payoff-based learning explains the decline in cooperation in public goods games. Proc R Soc Lond Ser B 282:20142678CrossRef

Dawes RM (1980) Social dilemmas. Annu Rev Psychol 31:169–193CrossRef

Dercole F, DeCarli M, Della Rossa F, Papadopoulos AV (2013) Overpunishing is not necessary to fix cooperation in voluntary public goods games. J Theor Biol 326:70–81CrossRef

Efferson C, Roca CP, Vogt S, Helbing D (2016) Sustained cooperation by running away from bad behavior. Evol Hum Behav 37(1):1–9CrossRef

Egas M, Riedl A (2008) The economics of altruistic punishment and the maintenance of cooperation. Proc Biol Sci 275:871–878CrossRef

Fehr E, Gachter S (2002) Altruistic punishment in humans. Nature 415:137–140CrossRef

Fowler JH (2005) Altruistic punishment and the origin of cooperation. Proc Natl Acad Sci 102:7047–7049CrossRef

Gómez-Gardeñes J, Poncela J, Floria LM, Moreno YJ (2008) Natural selection of cooperation and degree hierarchy in heterogeneous populations. J Theor Biol 253:296–301CrossRef

Hardin G (1968) The tragedy of the commons. Science 162:1243–1248CrossRef

Hauert C, Schuster P (1998) Extending the iterated prisoner’s dilemma without synchrony. J Theor Biol 192(2):155–166CrossRef

Hauert C, Traulsen A, Brandt H, Nowak MA, Sigmund K (2007) Via freedom to coercion: the emergence of costly punishment. Science 316:1905–1907CrossRef

Hauert C, Traulsen A, Brandt H, Nowak MA, Sigmund K (2008) Public goods with punishment and abstaining in finite and infinite populations. Biol Theory 3(2):114–122CrossRef

Kagel J, Roth A (1997) The handbook of experimental economics. Princeton University Press, Princeton

Kianercy A, Galstyan A (2012) Dynamics of Boltzmann Q learning in two-player two-action games. Phys Rev E 85:041145CrossRef

Lewin AY, Long CP, Carroll TN (1999) The coevolution of new organizational forms. Organ Sci 10(5):535–550CrossRef

Nakamaru M, Dieckmann U (2009) Runaway selection for cooperation and strict-and-severe punishment. J Theor Biol 257(1):1–8CrossRef

Nax HH, Perc M (2015) Directional learning and the provisioning of public goods. Sci Rep 5:8010CrossRef

Nax HH, Perc M, Szolnoki A, Helbing D (2015) Stability of cooperation under image scoring in group interactions. Sci Rep 5:12145CrossRef

Nax HH, Burton-Chellew MN, West SA, Young PH (2016) Learning in a black box. J Econ Behav Organ 127:1–15CrossRef

Perc M, Gómez-Gardeñes J, Szolnoki A, Floría LM, Moreno Y (2013) Evolutionary dynamics of group interactions on structured populations: a review. J R Soc Interface 10:20120997CrossRef

Raub W, Snijders C (1997) Gains, losses and cooperation in social dilemmas and collective action: the effects of risk preferences. J Math Sociol 22:263–291CrossRef

Santos FC, Pacheco JM (2005) Scale-free networks provide a unifying framework for the emergence of cooperation. Phys Rev Lett 95:098104CrossRef

Santos FC, Santos MD, Pacheco JM (2008) Social diversity promotes the emergence of cooperation in public goods games. Nature 454:213–216CrossRef

Sasaki T, Brannstrom A, Dieckmanna U, Sigmund K (2012) The take-it-or-leave-it option allows small penalties to overcome social dilemmas. Proc Natl Sci USA 109:1165–1169CrossRef

Schuster P, Sigmund K (1983) Replicator dynamics. J Theor Biol 100:533–538CrossRef

Sigmund K, DeSilva H, Traulsen A, Hauert C (2010) Social learning promotes institutions for governing the commons. Nature 466:861–863CrossRef

Solferino N, Taurino SF (2016) The evolution of cooperation with punishment and second chance. Homo Oecon 32(3/4):387–400

Strogatz SH (2001) Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. Westview Press, Boulder

Van Assen MALM, Snijders C (2004) Effects of risk preferences in social dilemmas: a game-theoretical analysis and evidence from two experiments. Cambridge University Press, Cambridge

Waltman L, Kaymak U (2008) \(Q\)-learning agents in a Cournot oligopoly model. J Econ Dyn Control 32(10):3275–3293CrossRef

West SA, Griffin AS, Gardner A (2007) Evolutionary explanations for cooperation. Curr Biol 17:R661–R672CrossRef

Xie MC, Tachibana A (2007) Cooperative behavior acquisition for multi-agent systems by Q-learning. In: Proceeding of the 2007 IEEE symposium on foundations of computational intelligence, FOCI 2007, 1–5 April 2007. IEEE, pp 424–428

Young HP (2009) Learning by trial and error. Games Econ Behav 65:626–643CrossRef

Titel: The economics analysis of a Q-learning model of cooperation with punishment and risk taking preferences
verfasst von: Nazaria Solferino
Viviana Solferino
Serena F. Taurino
Publikationsdatum: 09.04.2017
Verlag: Springer Berlin Heidelberg
Erschienen in: Journal of Economic Interaction and Coordination / Ausgabe 3/2018
Print ISSN: 1860-711X
Elektronische ISSN: 1860-7128
DOI: https://doi.org/10.1007/s11403-017-0195-2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 3/2018

Revisiting the issue of survivability and market efficiency with the Santa Fe Artificial Stock Market

Artificial stock markets with different maturity levels: simulation of information asymmetry and herd behavior using agent-based and network models

Estimating heterogeneous agents behavior in a two-market financial system

An agent-based model of the observed distribution of wealth in the United States

Quantifying invariant features of within-group inequality in consumption across groups

The creative response and the endogenous dynamics of pecuniary knowledge externalities: an agent based simulation model