Skip to main content
Top
Published in: Neural Computing and Applications 1/2017

28-05-2016 | Original Article

An efficient trust estimation model for multi-agent systems using temporal difference learning

Authors: G. Rishwaraj, S. G. Ponnambalam, Loo Chu Kiong

Published in: Neural Computing and Applications | Special Issue 1/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In multi-agent system (MAS) applications, teamwork among the agents is essential as the agents are required to collaborate and pool resources to execute the given tasks and complete the objectives successfully. A vital part of the collaboration is sharing of information and resources in order to optimize their efforts in achieving the given objectives. Under such collaborative environment, trust among the agents plays a critical role to ensure efficient cooperation. This study looks into developing a trust evaluation model that can empirically evaluate the trust of one agent on the other. The proposed model is developed using temporal difference learning method, incorporating experience gained through interactions into trust evaluation. Simulation experiments are conducted to evaluate the performance of the developed model against some of the most recent models reported in the literature. The results of the simulation experiments indicate that the proposed model performs better than the comparison models in estimating trust more effectively.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Basheer GS, Ahmad MS, Tang AY, Graf S (2015) Certainty, trust and evidence: towards an integrative model of confidence in multi-agent systems. Comput Hum Behav 45:307–315CrossRef Basheer GS, Ahmad MS, Tang AY, Graf S (2015) Certainty, trust and evidence: towards an integrative model of confidence in multi-agent systems. Comput Hum Behav 45:307–315CrossRef
2.
go back to reference Yugang L, Nejat G, Vilela J (2013) ‘Learning to cooperate together: a semi-autonomous control architecture for multi-robot teams in urban search and rescue’, Safety, Security, and Rescue Robotics (SSRR), 2013 IEEE International Symposium on, pp 1–6 Yugang L, Nejat G, Vilela J (2013) ‘Learning to cooperate together: a semi-autonomous control architecture for multi-robot teams in urban search and rescue’, Safety, Security, and Rescue Robotics (SSRR), 2013 IEEE International Symposium on, pp 1–6
3.
go back to reference Mohammed M, Lim C, Quteishat A (2014) A novel trust measurement method based on certified belief in strength for a multi-agent classifier system. Neural Comput and Appl 24(2):421–429CrossRef Mohammed M, Lim C, Quteishat A (2014) A novel trust measurement method based on certified belief in strength for a multi-agent classifier system. Neural Comput and Appl 24(2):421–429CrossRef
4.
go back to reference Fullam KK, Klos TB, Muller G, Sabater J, Schlosser A, Topol Z, Barber KS, Rosenschein JS, Vercouter L, Voss M (2005) ‘A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies’, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, pp 512–518 Fullam KK, Klos TB, Muller G, Sabater J, Schlosser A, Topol Z, Barber KS, Rosenschein JS, Vercouter L, Voss M (2005) ‘A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies’, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, pp 512–518
5.
go back to reference Han Y, Zhiqi S, Leung C, Chunyan M, Lesser VR (2013) A survey of multi-agent trust management systems. IEEE Access 1:35–50CrossRef Han Y, Zhiqi S, Leung C, Chunyan M, Lesser VR (2013) A survey of multi-agent trust management systems. IEEE Access 1:35–50CrossRef
6.
go back to reference Busoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C Appl Rev 38(2):156–172CrossRef Busoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C Appl Rev 38(2):156–172CrossRef
7.
go back to reference Aref A, Tran T (2014) ‘Using fuzzy logic and Q-learning for trust modeling in multi-agent systems’, Computer Science and Information Systems (FedCSIS), 2014 Federated Conference on, pp 59–66 Aref A, Tran T (2014) ‘Using fuzzy logic and Q-learning for trust modeling in multi-agent systems’, Computer Science and Information Systems (FedCSIS), 2014 Federated Conference on, pp 59–66
8.
go back to reference Das A, Islam MM (2012) SecuredTrust: a dynamic trust computation model for secured communication in multiagent systems. IEEE Trans Dependable Secure Comput 9(2):261–274CrossRef Das A, Islam MM (2012) SecuredTrust: a dynamic trust computation model for secured communication in multiagent systems. IEEE Trans Dependable Secure Comput 9(2):261–274CrossRef
10.
go back to reference Kurniawati H, Yanzhu D, Hsu D, Lee WS (2010) Motion planning under uncertainty for robotic tasks with long time horizons. Int J Robot Res 30(3):308–323CrossRefMATH Kurniawati H, Yanzhu D, Hsu D, Lee WS (2010) Motion planning under uncertainty for robotic tasks with long time horizons. Int J Robot Res 30(3):308–323CrossRefMATH
11.
go back to reference Kurniawati H, Bandyopadhyay T, Patrikalakis NM (2012) Global motion planning under uncertain motion, sensing, and environment map. Auton Robots 33(3):255–272CrossRef Kurniawati H, Bandyopadhyay T, Patrikalakis NM (2012) Global motion planning under uncertain motion, sensing, and environment map. Auton Robots 33(3):255–272CrossRef
12.
go back to reference Gorbunov R, Barakova E, Rauterberg M (2013) Design of social agents. Neurocomputing 114:92–97CrossRef Gorbunov R, Barakova E, Rauterberg M (2013) Design of social agents. Neurocomputing 114:92–97CrossRef
13.
go back to reference Rosaci D, Sarné GML, Garruzzo S (2012) Integrating trust measures in multiagent systems. Int J Intell Syst 27(1):1–15CrossRef Rosaci D, Sarné GML, Garruzzo S (2012) Integrating trust measures in multiagent systems. Int J Intell Syst 27(1):1–15CrossRef
14.
go back to reference Sabater J, Sierra C (2005) Review on computational trust and reputation models. Artif Intell Rev 24(1):33–60CrossRefMATH Sabater J, Sierra C (2005) Review on computational trust and reputation models. Artif Intell Rev 24(1):33–60CrossRefMATH
15.
go back to reference Vanderelst D, Ahn RM, Barakova EI (2008) Simulated Trust-Towards robust social learning. In: ALIFE, pp 632–639 Vanderelst D, Ahn RM, Barakova EI (2008) Simulated Trust-Towards robust social learning. In: ALIFE, pp 632–639
16.
go back to reference Vanderelst D, Ahn RM, Barakova EI (2009) Simulated trust: a cheap social learning strategy. Theor Popul Biol 76(3):189–196CrossRefMATH Vanderelst D, Ahn RM, Barakova EI (2009) Simulated trust: a cheap social learning strategy. Theor Popul Biol 76(3):189–196CrossRefMATH
17.
go back to reference Huynh TD, Jennings NR, Shadbolt NR (2006) An integrated trust and reputation model for open multi-agent systems. Auton Agent Multi-Agent Syst 13(2):119–154CrossRef Huynh TD, Jennings NR, Shadbolt NR (2006) An integrated trust and reputation model for open multi-agent systems. Auton Agent Multi-Agent Syst 13(2):119–154CrossRef
18.
go back to reference Yu H, Shen Z, Miao C, An B, Leung C (2014) Filtering trust opinions through reinforcement learning. Decis Support Syst 66:102–113CrossRef Yu H, Shen Z, Miao C, An B, Leung C (2014) Filtering trust opinions through reinforcement learning. Decis Support Syst 66:102–113CrossRef
20.
go back to reference Zhou P, Gu X, Zhang J, Fei M (2015) A priori trust inference with context-aware stereotypical deep learning. Knowl-Based Syst 88:97–106CrossRef Zhou P, Gu X, Zhang J, Fei M (2015) A priori trust inference with context-aware stereotypical deep learning. Knowl-Based Syst 88:97–106CrossRef
21.
go back to reference Fan W, Perros H (2014) A novel trust management framework for multi-cloud environments based on trust service providers. Knowl-Based Syst 70:392–406CrossRef Fan W, Perros H (2014) A novel trust management framework for multi-cloud environments based on trust service providers. Knowl-Based Syst 70:392–406CrossRef
22.
go back to reference Jin-Hee C, Chan K, Mikulski D (2014) Trust-based information and decision fusion for military convoy operations, Military Communications Conference (MILCOM), 2014 IEEE, pp 1387–1392 Jin-Hee C, Chan K, Mikulski D (2014) Trust-based information and decision fusion for military convoy operations, Military Communications Conference (MILCOM), 2014 IEEE, pp 1387–1392
23.
go back to reference Wang Y, Singh MP (2007) Formal trust model for multiagent systems, In Proceedings of the 20th international joint conference on Artifical intelligence, pp 1551–1556 Wang Y, Singh MP (2007) Formal trust model for multiagent systems, In Proceedings of the 20th international joint conference on Artifical intelligence, pp 1551–1556
24.
go back to reference Wang Y, Singh MP (2010) Evidence-based trust: a mathematical model geared for multiagent systems. ACM Trans Auton Adapt Syst 5(4):1–28CrossRef Wang Y, Singh MP (2010) Evidence-based trust: a mathematical model geared for multiagent systems. ACM Trans Auton Adapt Syst 5(4):1–28CrossRef
25.
go back to reference Wang Y, Hang C-W, Singh MP (2011) A probabilistic approach for maintaining Trust.pdf. J Artif Intell Res 40:47 Wang Y, Hang C-W, Singh MP (2011) A probabilistic approach for maintaining Trust.pdf. J Artif Intell Res 40:47
26.
go back to reference Mantel KT, Clark CM (2012) Trust networks in multi-robot communities, In Robotics and Biomimetics (ROBIO), 2012 IEEE International Conference on, pp 2114–2119 Mantel KT, Clark CM (2012) Trust networks in multi-robot communities, In Robotics and Biomimetics (ROBIO), 2012 IEEE International Conference on, pp 2114–2119
27.
go back to reference Jøsang A, Haller J (2007) ‘Dirichlet Reputation Systems’, Availability, Reliability and Security, ARES 2007. The Second International Conference on, pp 112–119 Jøsang A, Haller J (2007) ‘Dirichlet Reputation Systems’, Availability, Reliability and Security, ARES 2007. The Second International Conference on, pp 112–119
28.
go back to reference Namin AS, Ruizhong W, Weiming S, Ghenniwa H (2006) An efficient trust model for multi-agent systems, Computer Supported Cooperative Work in Design, 2006. CSCWD ‘06. 10th International Conference on, pp 1–6 Namin AS, Ruizhong W, Weiming S, Ghenniwa H (2006) An efficient trust model for multi-agent systems, Computer Supported Cooperative Work in Design, 2006. CSCWD ‘06. 10th International Conference on, pp 1–6
29.
go back to reference Peng M, Xu Z, Pan S, Li R, Mao T (2012) AgentTMS: a MAS trust model based on agent social relationship. J Comput 7(6):1535–1542CrossRef Peng M, Xu Z, Pan S, Li R, Mao T (2012) AgentTMS: a MAS trust model based on agent social relationship. J Comput 7(6):1535–1542CrossRef
30.
go back to reference Mui L, Mohtashemi M, Halberstadt A (2002) ‘A computational model of trust and reputation’, System Sciences, 2002. HICSS. Proceedings of the 35th Annual Hawaii International Conference on, pp 2431–2439 Mui L, Mohtashemi M, Halberstadt A (2002) ‘A computational model of trust and reputation’, System Sciences, 2002. HICSS. Proceedings of the 35th Annual Hawaii International Conference on, pp 2431–2439
31.
go back to reference Griffiths N (2005) Task delegation using experience-based multi-dimensional trust, In Proceedings of the fourth international joint conference on autonomous agents and multiagent systems, pp 489-496 Griffiths N (2005) Task delegation using experience-based multi-dimensional trust, In Proceedings of the fourth international joint conference on autonomous agents and multiagent systems, pp 489-496
32.
go back to reference Fullam KK, Barber KS (2007) Dynamically learning sources of trust information: experience versus reputation, In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, pp 1–8 Fullam KK, Barber KS (2007) Dynamically learning sources of trust information: experience versus reputation, In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, pp 1–8
33.
go back to reference Russell SJ, Norvig P (2003) Artificial intelligence a modern approach (Pearson Education, Inc., 2003, Second edn. 2003) Russell SJ, Norvig P (2003) Artificial intelligence a modern approach (Pearson Education, Inc., 2003, Second edn. 2003)
34.
go back to reference Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT press, Cambridge Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT press, Cambridge
35.
go back to reference Jsang A, Ismail R (2002) The beta reputation system, In Proceedings of the 15th bled electronic commerce conference, vol 5, pp 2502–2511 Jsang A, Ismail R (2002) The beta reputation system, In Proceedings of the 15th bled electronic commerce conference, vol 5, pp 2502–2511
36.
go back to reference Gautam A, Mohan S (2012) A review of research in multi-robot systems’, Industrial and Information Systems (ICIIS), 2012 7th IEEE International Conference on, pp 1–5 Gautam A, Mohan S (2012) A review of research in multi-robot systems’, Industrial and Information Systems (ICIIS), 2012 7th IEEE International Conference on, pp 1–5
37.
go back to reference Nagatani K, Kiribayashi S, Okada Y, Otake K, Yoshida K, Tadokoro S, Nishimura T, Yoshida T, Koyanagi E, Fukushima M, Kawatsuma S (2013) Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots. J Field Robot 30(1):44–63CrossRef Nagatani K, Kiribayashi S, Okada Y, Otake K, Yoshida K, Tadokoro S, Nishimura T, Yoshida T, Koyanagi E, Fukushima M, Kawatsuma S (2013) Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots. J Field Robot 30(1):44–63CrossRef
38.
go back to reference Burdakov O, Doherty P, Holmberg K, Kvarnstrom J, Olsson PM (2010) Relay positioning for unmanned aerial vehicle surveillance. Int J Robot Res 29(8):1069–1087CrossRef Burdakov O, Doherty P, Holmberg K, Kvarnstrom J, Olsson PM (2010) Relay positioning for unmanned aerial vehicle surveillance. Int J Robot Res 29(8):1069–1087CrossRef
Metadata
Title
An efficient trust estimation model for multi-agent systems using temporal difference learning
Authors
G. Rishwaraj
S. G. Ponnambalam
Loo Chu Kiong
Publication date
28-05-2016
Publisher
Springer London
Published in
Neural Computing and Applications / Issue Special Issue 1/2017
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-016-2354-0

Other articles of this Special Issue 1/2017

Neural Computing and Applications 1/2017 Go to the issue

Premium Partner