Skip to main content
Erschienen in: Information Systems Frontiers 4/2015

01.08.2015

On the inaccuracy of numerical ratings: dealing with biased opinions in social networks

verfasst von: Roberto Centeno, Ramón Hermoso, Maria Fasli

Erschienen in: Information Systems Frontiers | Ausgabe 4/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this work, we study the potential problems emanating from using numerical ratings in social networks to rank entities regarding their reputation. In particular, we empirically demonstrate how reputation rankings as collected and managed by current systems are likely to be skewed due to subjectivity problems associated with the use of numerical ratings to encapsulate preferences. With the aim of overcoming these problems, we put forward an approach in which users are asked for their opinions about entities in a comparative fashion. In order to select the most appropriate users to be queried, we take advantage of the social structure derived from the interactions among users and entities following a principle of heterogeneity. Finally, we evaluate the proposed approach in the domain of movie ratings by using real datasets collected from different web sites.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
4
This dataset was used in the 2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems (See http://​ir.​ii.​uam.​es/​hetrec2011). It is collected from the MovieLens opinion site, where movies are rated by different users.
 
5
As in the MovieLens site, this is an opinion website where movies are rated. However, in this case, movies are also reviewed by professional critics (See http://​www.​rottentomatoes.​com).
 
Literatur
Zurück zum Zitat Anderson, M.L., & Magruder, J.R. (2012). Learning from the crowd: Regression discontinuity estimates of the effects of an online review database. Economic Journal, 122 (563), 957–989.CrossRef Anderson, M.L., & Magruder, J.R. (2012). Learning from the crowd: Regression discontinuity estimates of the effects of an online review database. Economic Journal, 122 (563), 957–989.CrossRef
Zurück zum Zitat Balakrishnan, S., & Chopra, S. (2010). Two of a kind or the ratings game? Adaptive pairwise preferences and latent factor models. Proceedings of the 2010 IEEE international conference on data mining. ICDM ’10, (pp. 725–730). Washington, DC: IEEE Computer Society. Balakrishnan, S., & Chopra, S. (2010). Two of a kind or the ratings game? Adaptive pairwise preferences and latent factor models. Proceedings of the 2010 IEEE international conference on data mining. ICDM ’10, (pp. 725–730). Washington, DC: IEEE Computer Society.
Zurück zum Zitat Berinsky, A.J. (1999). The two faces of public opinion. American Journal of Political Science, 43 (4), 1209–1230.CrossRef Berinsky, A.J. (1999). The two faces of public opinion. American Journal of Political Science, 43 (4), 1209–1230.CrossRef
Zurück zum Zitat Brams, S.J., & Fishburn, P.C. (2002). Voting procedures In Arrow, K. J., Sen, A. K., Suzumura, K. (Eds.), Handbook of social choice and welfare (Vol. 1, pp. 173–236). New York: Elsevier. Brams, S.J., & Fishburn, P.C. (2002). Voting procedures In Arrow, K. J., Sen, A. K., Suzumura, K. (Eds.), Handbook of social choice and welfare (Vol. 1, pp. 173–236). New York: Elsevier.
Zurück zum Zitat Burke, R. (2000). Knowledge-based Recommender Systems. Encyclopedia of library and information systems Vol. 69. Burke, R. (2000). Knowledge-based Recommender Systems. Encyclopedia of library and information systems Vol. 69.
Zurück zum Zitat Chen, Li, & Pu, Pearl (2004). Survey of preference elicitation methods. Tech. rep., Swiss Federal Institure of Technology in Lausanne (EPFL). Chen, Li, & Pu, Pearl (2004). Survey of preference elicitation methods. Tech. rep., Swiss Federal Institure of Technology in Lausanne (EPFL).
Zurück zum Zitat Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, J. (2003). Is seeing believing?: how recommender system interfaces affect users’ opinions. Proceedings of the sigchi conference on human factors in computing systems. CHI ’03, (pp. 585–592). New York: ACM. Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, J. (2003). Is seeing believing?: how recommender system interfaces affect users’ opinions. Proceedings of the sigchi conference on human factors in computing systems. CHI ’03, (pp. 585–592). New York: ACM.
Zurück zum Zitat Dalvi, N., Kumar, R., Pang, B. (2013). Para ’normal’ activity: On the distribution of average ratings. Proceedings of 7th international AAAI conference on weblogs and social media, (pp. 110–119). Dalvi, N., Kumar, R., Pang, B. (2013). Para ’normal’ activity: On the distribution of average ratings. Proceedings of 7th international AAAI conference on weblogs and social media, (pp. 110–119).
Zurück zum Zitat Dellarocas, C. (2000). Mechanisms for coping with unfair ratings and discriminatory behavior in online reputation reporting systems. ICIS ’00: Proceedings of the 21st international conference on information systems, (pp. 520–525). Atlanta: Association for Information Systems. Dellarocas, C. (2000). Mechanisms for coping with unfair ratings and discriminatory behavior in online reputation reporting systems. ICIS ’00: Proceedings of the 21st international conference on information systems, (pp. 520–525). Atlanta: Association for Information Systems.
Zurück zum Zitat Fowler, J.H., & Christakis, N.A. (2010). Cooperative behavior cascades in human social networks. Proceedings of the National Academy of Sciences, 107 (12), 5334–5338.CrossRef Fowler, J.H., & Christakis, N.A. (2010). Cooperative behavior cascades in human social networks. Proceedings of the National Academy of Sciences, 107 (12), 5334–5338.CrossRef
Zurück zum Zitat Fu, F., Hauert, C., Nowak, M.A., Wang, L. (2008). Reputation-based partner choice promotes cooperation in social networks. Physical Review E, 78. Fu, F., Hauert, C., Nowak, M.A., Wang, L. (2008). Reputation-based partner choice promotes cooperation in social networks. Physical Review E, 78.
Zurück zum Zitat Fullam, K.K., & Barber, K.S. (2007). Dynamically learning sources of trust information: experience vs. reputation. Proceedings of AAMAS ’07, (pp. 1055–1062). New York: ACM. Fullam, K.K., & Barber, K.S. (2007). Dynamically learning sources of trust information: experience vs. reputation. Proceedings of AAMAS ’07, (pp. 1055–1062). New York: ACM.
Zurück zum Zitat Grinstead, C.M., & Snell, J.L. (1997). Introduction to probability, 2nd edn, AMS Bookstore, chap Central Limit Theorem, (pp. 325–360). Grinstead, C.M., & Snell, J.L. (1997). Introduction to probability, 2nd edn, AMS Bookstore, chap Central Limit Theorem, (pp. 325–360).
Zurück zum Zitat Hermoso, R., Centeno, R., Fasli, M. (2013). From blurry numbers to clear preferences: A mechanism to extract reputation in social networks. Expert Systems with Applications, 41 (5), 2269–2285.CrossRef Hermoso, R., Centeno, R., Fasli, M. (2013). From blurry numbers to clear preferences: A mechanism to extract reputation in social networks. Expert Systems with Applications, 41 (5), 2269–2285.CrossRef
Zurück zum Zitat Hill, W., Stead, L., Rosenstein, M., Furnas, G. (1995). Recommending and evaluating choices in a virtual community of use. Proceedings of the sigchi conference on human factors in computing systems. CHI ’95, (pp. 194–201). New York: ACM Press/Addison-Wesley Publishing Co. Hill, W., Stead, L., Rosenstein, M., Furnas, G. (1995). Recommending and evaluating choices in a virtual community of use. Proceedings of the sigchi conference on human factors in computing systems. CHI ’95, (pp. 194–201). New York: ACM Press/Addison-Wesley Publishing Co.
Zurück zum Zitat Huynh, T.D., Jennings, N.R., Shadbolt, N.R. (2006). An integrated trust and reputation model for open multi-agent systems. Autonomous Agents and Multi-Agent Systems, 13 (2), 119–154.CrossRef Huynh, T.D., Jennings, N.R., Shadbolt, N.R. (2006). An integrated trust and reputation model for open multi-agent systems. Autonomous Agents and Multi-Agent Systems, 13 (2), 119–154.CrossRef
Zurück zum Zitat Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems, 20 (4), 422–446.CrossRef Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems, 20 (4), 422–446.CrossRef
Zurück zum Zitat Jøsang, A., & Ismail, R. (2002). The beta reputation system. Proceedings of the 15th bled electronic commerce conference. Jøsang, A., & Ismail, R. (2002). The beta reputation system. Proceedings of the 15th bled electronic commerce conference.
Zurück zum Zitat Jøsang, A., Ismail, R., Boyd, C. (2007). A survey of trust and reputation systems for online service provision. Decisions Support Systems, 43 (2), 618–644.CrossRef Jøsang, A., Ismail, R., Boyd, C. (2007). A survey of trust and reputation systems for online service provision. Decisions Support Systems, 43 (2), 618–644.CrossRef
Zurück zum Zitat Khosravifar, B., Bentahar, J., Gomrokchi, M., Alam, R. (2012). CRM: An efficient trust and reputation model for agent computing. Knowledge-Based Systems, 30 (0), 1 –16.CrossRef Khosravifar, B., Bentahar, J., Gomrokchi, M., Alam, R. (2012). CRM: An efficient trust and reputation model for agent computing. Knowledge-Based Systems, 30 (0), 1 –16.CrossRef
Zurück zum Zitat Kramer, M. (2007). Self-selection bias in reputation systems In Etalle, S., & Marsh, S. (Eds.), Trust management. IFIP International Federation for Information Processing (Vol. 238, pp. 255–268). Berlin Heildelberg New York: Springer. Kramer, M. (2007). Self-selection bias in reputation systems In Etalle, S., & Marsh, S. (Eds.), Trust management. IFIP International Federation for Information Processing (Vol. 238, pp. 255–268). Berlin Heildelberg New York: Springer.
Zurück zum Zitat Liu, Y., Li, K., Jin, Y., Zhang, Y., Qu, W. (2011). A novel reputation computation model based on subjective logic for mobile ad hoc networks. Future Generation Computer Systems, 27 (5), 547 –554.CrossRef Liu, Y., Li, K., Jin, Y., Zhang, Y., Qu, W. (2011). A novel reputation computation model based on subjective logic for mobile ad hoc networks. Future Generation Computer Systems, 27 (5), 547 –554.CrossRef
Zurück zum Zitat Luca, M. (2011). Reviews, reputation, and revenue: The case of yelp.com. Harvard Business School Working Papers 12–016, Harvard Business School. Luca, M. (2011). Reviews, reputation, and revenue: The case of yelp.com. Harvard Business School Working Papers 12–016, Harvard Business School.
Zurück zum Zitat Negahban, S., Oh, S., Shah, D. (2012). Iterative ranking from pair-wise comparisons. Proceedings of the 26th annual conference on neural information processing systems, (pp. 2483–2491). Negahban, S., Oh, S., Shah, D. (2012). Iterative ranking from pair-wise comparisons. Proceedings of the 26th annual conference on neural information processing systems, (pp. 2483–2491).
Zurück zum Zitat Pennock, D., Horvitz, E., Lawrence, S., Giles, C.L. (2000). Collaborative filtering by personality diagnosis: A hybrid memory- and model-based approach. Proceedings of the sixteenth conference on uncertainty in artificial intelligence, (pp. 473–480). San Mateo: Morgan Kaufmann. Pennock, D., Horvitz, E., Lawrence, S., Giles, C.L. (2000). Collaborative filtering by personality diagnosis: A hybrid memory- and model-based approach. Proceedings of the sixteenth conference on uncertainty in artificial intelligence, (pp. 473–480). San Mateo: Morgan Kaufmann.
Zurück zum Zitat Pinyol, I., & Sabater-Mir, J. (2013). Computational trust and reputation models for open multi-agent systems: a review. Artificial Intelligence Review, 40 (1), 1–25.CrossRef Pinyol, I., & Sabater-Mir, J. (2013). Computational trust and reputation models for open multi-agent systems: a review. Artificial Intelligence Review, 40 (1), 1–25.CrossRef
Zurück zum Zitat Ramchurn, S.D., Huynh, D., Jennings, N.R. (2004). Trust in multi-agent systems. Knowledge Engineering Review, 19 (1), 1–25.CrossRef Ramchurn, S.D., Huynh, D., Jennings, N.R. (2004). Trust in multi-agent systems. Knowledge Engineering Review, 19 (1), 1–25.CrossRef
Zurück zum Zitat Resnick, P., Kuwabara, K., Zeckhauser, R., Friedman, E. (2000). Reputation systems. Communications of the ACM, 43 (12), 45–48.CrossRef Resnick, P., Kuwabara, K., Zeckhauser, R., Friedman, E. (2000). Reputation systems. Communications of the ACM, 43 (12), 45–48.CrossRef
Zurück zum Zitat Shearin, S., & Lieberman, H. (2001). Intelligent profiling by example. Proceedings of the 6th international conference on intelligent user interfaces. IUI ’01, (pp. 145–151). New York: ACM. Shearin, S., & Lieberman, H. (2001). Intelligent profiling by example. Proceedings of the 6th international conference on intelligent user interfaces. IUI ’01, (pp. 145–151). New York: ACM.
Zurück zum Zitat Sierra, C., & Sabater, J. (2005). Review on computionational trust and reputation models. Artificial Intelligence Review, 24 (1), 33 –60.CrossRef Sierra, C., & Sabater, J. (2005). Review on computionational trust and reputation models. Artificial Intelligence Review, 24 (1), 33 –60.CrossRef
Zurück zum Zitat Sparling, E.I., & Sen, S. (2011). Rating: how difficult is it?. Proceedings of the 5th acm conference on recommender systems. RecSys ’11, (pp. 149–156). New York: ACM. Sparling, E.I., & Sen, S. (2011). Rating: how difficult is it?. Proceedings of the 5th acm conference on recommender systems. RecSys ’11, (pp. 149–156). New York: ACM.
Zurück zum Zitat Tanimoto, T. (1958). An elementary mathematical theory of classification and prediction. IBM Internal Report. Tanimoto, T. (1958). An elementary mathematical theory of classification and prediction. IBM Internal Report.
Zurück zum Zitat Teacy, W.T.L., Patel, J., Jennings, N.R., Luck, M. (2006). Travos: Trust and reputation in the context of inaccurate information sources. Autonomous Agents and Multi-Agent Systems, 12 (2), 183–198.CrossRef Teacy, W.T.L., Patel, J., Jennings, N.R., Luck, M. (2006). Travos: Trust and reputation in the context of inaccurate information sources. Autonomous Agents and Multi-Agent Systems, 12 (2), 183–198.CrossRef
Zurück zum Zitat Teacy, W.T.L., Luck, M., Rogers, A., Jennings, N.R. (2012). An efficient and versatile approach to trust and reputation using hierarchical bayesian modelling. Artificial Intelligence, 193, 149–185.CrossRef Teacy, W.T.L., Luck, M., Rogers, A., Jennings, N.R. (2012). An efficient and versatile approach to trust and reputation using hierarchical bayesian modelling. Artificial Intelligence, 193, 149–185.CrossRef
Zurück zum Zitat Yu, B., & Singh, M.P. (2002). An evidential model of distributed reputation management. AAMAS ’02: Proceedings of the first international joint conference on autonomous agents and multiagent systems, (pp. 294–301). New York: ACM Press. Yu, B., & Singh, M.P. (2002). An evidential model of distributed reputation management. AAMAS ’02: Proceedings of the first international joint conference on autonomous agents and multiagent systems, (pp. 294–301). New York: ACM Press.
Zurück zum Zitat Yu, B., Singh, M.P., Sycara, K. (2004). Developing trust in large-scale peer-to-peer systems. Proceedings of first IEEE symposium on multi-agent security and survivability, (pp. 1–10). Yu, B., Singh, M.P., Sycara, K. (2004). Developing trust in large-scale peer-to-peer systems. Proceedings of first IEEE symposium on multi-agent security and survivability, (pp. 1–10).
Metadaten
Titel
On the inaccuracy of numerical ratings: dealing with biased opinions in social networks
verfasst von
Roberto Centeno
Ramón Hermoso
Maria Fasli
Publikationsdatum
01.08.2015
Verlag
Springer US
Erschienen in
Information Systems Frontiers / Ausgabe 4/2015
Print ISSN: 1387-3326
Elektronische ISSN: 1572-9419
DOI
https://doi.org/10.1007/s10796-014-9526-1

Weitere Artikel der Ausgabe 4/2015

Information Systems Frontiers 4/2015 Zur Ausgabe

Premium Partner