Skip to main content
SpringerLink
Log in

TREET: the Trust and Reputation Experimentation and Evaluation Testbed

  • Published:
Electronic Commerce Research Aims and scope Submit manuscript

Abstract

To date, trust and reputation systems have often been evaluated using methods of their designers’ own devising. Recently, we demonstrated that a number of noteworthy trust and reputation systems could be readily defeated, revealing limitations in their original evaluations. Efforts in the trust and reputation community to develop a testbed have yielded a successful competition platform, ART. This testbed, however, is less suited to general experimentation and evaluation of individual trust and reputation technologies. In this paper, we present TREET, an experimentation and evaluation testbed based directly on that used in our investigations into security vulnerabilities in trust and reputation systems for marketplaces. We demonstrate the advantages of TREET, towards the development of more thorough, objective evaluations of trust and reputation systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fullam, K. K., Klos, T. B., Muller, G., Sabater, J., Schlosser, A., Topol, Z., Barber, K. S., Rosenschein, J. S., Vercouter, L., & Voss, M. (2005). A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies. In AAMAS ’05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems (pp. 512–518). New York: ACM.

    Chapter  Google Scholar 

  2. Teacy, W. T. L., Huynh, T. D., Dash, R. K., Jennings, N. R., Luck, M., & Patel, J. (2007). The ART of IAM: The winning strategy for the 2006 competition. In: Proceedings of the sixth international joint conference on autonomous agents and multiagent systems (AAMAS’07) workshop on trust in agent societies, Honolulu, Hawaii, USA.

  3. Teacy, W. T., Patel, J., Jennings, N. R., & Luck, M. (2006). Travos: trust and reputation in the context of inaccurate information sources. Autonomous Agents and Multi-Agent Systems, 12(2), 183–198.

    Article  Google Scholar 

  4. Yu, B., & Singh, M. P. (2002). Distributed reputation management for electronic commerce. Computational Intelligence, 18(4), 535–549.

    Article  Google Scholar 

  5. Kerr, R., & Cohen, R. (2006). Modeling trust using transactional, numerical units. In: PST ’06: proceedings of the conference on privacy, security and trust, Markham, Canada.

  6. Kerr, R., & Cohen, R. (2009). Smart cheaters do prosper: defeating trust and reputation systems. In: Proceedings of the eighth international joint conference on autonomous agents and multiagent systems (AAMAS’09), Budapest, Hungary.

  7. Kerr, R., & Cohen, R. (2009). An experimental testbed for evaluation of trust and reputation systems. In: Ferrari, E., Li, N., Bertino, E., Karabulut, Y. (Eds.), Trust management III. Advances in information and communication technology, vol. 300 (pp. 252–266). Boston: Springer.

    Chapter  Google Scholar 

  8. Jøsang, A., & Ismail, R. (2002). The beta reputation system. In: Proceedings of the 15th bled electronic commerce conference e-reality: constructing the e-economy, June 2002.

  9. Tran, T., & Cohen, R. (2004). Improving user satisfaction in agent-based electronic marketplaces by reputation modelling and adjustable product quality. In AAMAS ’04: proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 828–835). Washington: IEEE Computer Society.

    Google Scholar 

  10. Hang, C. W., Wang, Y., & Singh, M. P. (2008). An adaptive probabilistic trust model and its evaluation. In AAMAS ’08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems (pp. 1485–1488). Richland: International Foundation for Autonomous Agents and Multiagent Systems.

    Google Scholar 

  11. Harbers, M., Verbrugge, R., Sierra, C., & Debenham, J. (2008). The examination of an information-based approach to trust. In Noriega, P., Padget, J. (Eds.), International workshop on coordination, organization, institutions and norms (COIN) (pp. 101–112). Durham: Durham University.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada

    Reid Kerr & Robin Cohen

Authors
  1. Reid Kerr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robin Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reid Kerr.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kerr, R., Cohen, R. TREET: the Trust and Reputation Experimentation and Evaluation Testbed. Electron Commer Res 10, 271–290 (2010). https://doi.org/10.1007/s10660-010-9056-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10660-010-9056-y

Keywords

Search

Navigation