Skip to main content

2015 | OriginalPaper | Buchkapitel

Hierarchical Learning for Emergence of Social Norms in Networked Multiagent Systems

verfasst von : Chao Yu, Hongtao Lv, Fenghui Ren, Honglin Bao, Jianye Hao

Erschienen in: AI 2015: Advances in Artificial Intelligence

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, a hierarchical learning framework is proposed for emergence of social norms in networked multiagent systems. This framework features a bottom level of agents and several levels of supervisors. Agents in the bottom level interact with each other using reinforcement learning methods, and report their information to their supervisors after each interaction. Supervisors then aggregate the reported information and produce guide policies by exchanging information with other supervisors. The guide policies are then passed down to the subordinate agents in order to adjust their learning behaviors heuristically. Experiments are carried out to explore the efficiency of norm emergence under the proposed framework, and results verify that learning from local interactions integrating hierarchical supervision can be an effective mechanism for emergence of social norms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Savarimuthu, B.T.R., Arulanandam, R., Purvis, M.: Aspects of active norm learning and the effect of lying on norm emergence in agent societies. In: Kinny, D., Hsu, J.Y., Governatori, G., Ghose, A.K. (eds.) PRIMA 2011. LNCS, vol. 7047, pp. 36–50. Springer, Heidelberg (2011) CrossRef Savarimuthu, B.T.R., Arulanandam, R., Purvis, M.: Aspects of active norm learning and the effect of lying on norm emergence in agent societies. In: Kinny, D., Hsu, J.Y., Governatori, G., Ghose, A.K. (eds.) PRIMA 2011. LNCS, vol. 7047, pp. 36–50. Springer, Heidelberg (2011) CrossRef
2.
Zurück zum Zitat Mukherjee, P., Sen, S., Airiau, S.: Norm emergence under constrained interactions in diverse societies. In: Proceedings of 7th AAMAS, pp. 779–786 (2008) Mukherjee, P., Sen, S., Airiau, S.: Norm emergence under constrained interactions in diverse societies. In: Proceedings of 7th AAMAS, pp. 779–786 (2008)
3.
Zurück zum Zitat Hao, J., Sun, J., Huang, D., Cai, Y., Yu, C.: Heuristic collective learning for efficient and robust emergence of social norms. In: Proceedings of 14th AAMAS, pp. 1647–1648 (2015) Hao, J., Sun, J., Huang, D., Cai, Y., Yu, C.: Heuristic collective learning for efficient and robust emergence of social norms. In: Proceedings of 14th AAMAS, pp. 1647–1648 (2015)
4.
Zurück zum Zitat Yu, C., Zhang, M., Ren, F.: Collective learning for the emergence of social norms in networked multiagent systems. IEEE Trans. Cybernet. 44(12), 2342–2355 (2014)CrossRef Yu, C., Zhang, M., Ren, F.: Collective learning for the emergence of social norms in networked multiagent systems. IEEE Trans. Cybernet. 44(12), 2342–2355 (2014)CrossRef
5.
Zurück zum Zitat Yu, C., Zhang, M., Ren, F., Luo, X.: Emergence of social norms through collective learning in networked agent societies. In: Proceedings of the 12th AAMAS, pp. 475–482 (2013) Yu, C., Zhang, M., Ren, F., Luo, X.: Emergence of social norms through collective learning in networked agent societies. In: Proceedings of the 12th AAMAS, pp. 475–482 (2013)
6.
Zurück zum Zitat Yu, C., Zhang, M., Ren, F., Hao, J.: Emergence of social norms through collective learning in networked agent societies. In: Proceedings of MFSC@AAMAS 2014 (2014) Yu, C., Zhang, M., Ren, F., Hao, J.: Emergence of social norms through collective learning in networked agent societies. In: Proceedings of MFSC@AAMAS 2014 (2014)
7.
Zurück zum Zitat Villatoro, D., Sabater-Mir, J., Sen, S.: Social instruments for robust convention emergence. In: Proceedings of 22nd IJCAI, pp. 420–425 (2011) Villatoro, D., Sabater-Mir, J., Sen, S.: Social instruments for robust convention emergence. In: Proceedings of 22nd IJCAI, pp. 420–425 (2011)
8.
Zurück zum Zitat Shoham, Y., Tennenholtz, M.: On the emergence of social conventions: modeling, analysis, and simulations. Artif. Intell. 94(1), 139–166 (1997)CrossRefMATH Shoham, Y., Tennenholtz, M.: On the emergence of social conventions: modeling, analysis, and simulations. Artif. Intell. 94(1), 139–166 (1997)CrossRefMATH
9.
Zurück zum Zitat Young, H.P.: The economics of convention. J. Econ. Perspect. 10(2), 105–122 (1996)CrossRef Young, H.P.: The economics of convention. J. Econ. Perspect. 10(2), 105–122 (1996)CrossRef
10.
Zurück zum Zitat Sen, S., Airiau, S.: Emergence of norms through social learning. In: Proceedings of the 20nd IJCAI, pp. 1507–1512 (2007) Sen, S., Airiau, S.: Emergence of norms through social learning. In: Proceedings of the 20nd IJCAI, pp. 1507–1512 (2007)
11.
Zurück zum Zitat Sen, O., Sen, S.: Effects of social network topology and options on norm emergence. In: Padget, J., Artikis, A., Vasconcelos, W., Stathis, K., da Silva, V.T., Matson, E., Polleres, A. (eds.) COIN@AAMAS 2009. LNCS, vol. 6069, pp. 211–222. Springer, Heidelberg (2010)CrossRef Sen, O., Sen, S.: Effects of social network topology and options on norm emergence. In: Padget, J., Artikis, A., Vasconcelos, W., Stathis, K., da Silva, V.T., Matson, E., Polleres, A. (eds.) COIN@AAMAS 2009. LNCS, vol. 6069, pp. 211–222. Springer, Heidelberg (2010)CrossRef
12.
Zurück zum Zitat Sutton, R.S., Barto, A.G.: Reinforcement Learning: An introduction. MIT Press, Cambridge (1998) Sutton, R.S., Barto, A.G.: Reinforcement Learning: An introduction. MIT Press, Cambridge (1998)
13.
Zurück zum Zitat Weibull, J.W.: Evolutionary Game Theory. MIT Press, Cambridge (1997)MATH Weibull, J.W.: Evolutionary Game Theory. MIT Press, Cambridge (1997)MATH
14.
Zurück zum Zitat Pacheco, J.M., Traulsen, A., Nowak, M.A.: Coevolution of strategy and structure in complex networks with dynamical linking. Phys. Rev. Lett. 97(25), 1–9 (2006)CrossRef Pacheco, J.M., Traulsen, A., Nowak, M.A.: Coevolution of strategy and structure in complex networks with dynamical linking. Phys. Rev. Lett. 97(25), 1–9 (2006)CrossRef
15.
16.
Zurück zum Zitat Savarimuthu, B.: Norm Learning in Multi-agent Societies. University of Otago, Dunedin (2011) Savarimuthu, B.: Norm Learning in Multi-agent Societies. University of Otago, Dunedin (2011)
17.
Zurück zum Zitat Airiau, S., Sen, S., Villatoro, D.: Emergence of conventions through social learning. Auton. Agent. Multi-Agent Syst. 28(5), 779–804 (2014)CrossRef Airiau, S., Sen, S., Villatoro, D.: Emergence of conventions through social learning. Auton. Agent. Multi-Agent Syst. 28(5), 779–804 (2014)CrossRef
18.
Zurück zum Zitat Villatoro, D., Sen, S., Sabater-Mir. J.: Topology and memory effect on convention emergence. In: Proceedings of the 2009 WI and IAT, pp. 233–240 (2009) Villatoro, D., Sen, S., Sabater-Mir. J.: Topology and memory effect on convention emergence. In: Proceedings of the 2009 WI and IAT, pp. 233–240 (2009)
19.
Zurück zum Zitat Villatoro, D., Sabater-Mir, J., Sen, S.: Robust convention emergence in social networks through self-reinforcing structures dissolution. ACM Trans. Auton. Adapt. Syst. 8(1), 2–21 (2013)CrossRef Villatoro, D., Sabater-Mir, J., Sen, S.: Robust convention emergence in social networks through self-reinforcing structures dissolution. ACM Trans. Auton. Adapt. Syst. 8(1), 2–21 (2013)CrossRef
20.
Zurück zum Zitat Mihaylov, M., Tuyls, K., Now, A.: A decentralized approach for convention emergence in multi-agent systems. Auton. Agent. Multi-Agent Syst. 15(2), 1–30 (2013) Mihaylov, M., Tuyls, K., Now, A.: A decentralized approach for convention emergence in multi-agent systems. Auton. Agent. Multi-Agent Syst. 15(2), 1–30 (2013)
21.
Zurück zum Zitat Mahmoud, S., Griffiths, N., Keppens, J.: Norm emergence: overcoming hub effects in scale free networks. In: Proceedings of COIN, pp. 136–150 (2012) Mahmoud, S., Griffiths, N., Keppens, J.: Norm emergence: overcoming hub effects in scale free networks. In: Proceedings of COIN, pp. 136–150 (2012)
22.
Zurück zum Zitat Zhang, C., Sherief, A., Lesser, V.: Integrating organizational control into multi-agent learning. In: Proceedings of the 7th AAMAS, pp. 757–764 (2009) Zhang, C., Sherief, A., Lesser, V.: Integrating organizational control into multi-agent learning. In: Proceedings of the 7th AAMAS, pp. 757–764 (2009)
23.
Zurück zum Zitat Bianchi, R.A.C., Ribeiro, C.H.C., Costa, A.H.R.: Heuristic selection of actions in multiagent reinforcement learning. In: Proceedings of 2007 IJCAI, pp. 690–695 (2007) Bianchi, R.A.C., Ribeiro, C.H.C., Costa, A.H.R.: Heuristic selection of actions in multiagent reinforcement learning. In: Proceedings of 2007 IJCAI, pp. 690–695 (2007)
Metadaten
Titel
Hierarchical Learning for Emergence of Social Norms in Networked Multiagent Systems
verfasst von
Chao Yu
Hongtao Lv
Fenghui Ren
Honglin Bao
Jianye Hao
Copyright-Jahr
2015
DOI
https://doi.org/10.1007/978-3-319-26350-2_56