Sanjay Krishnan
ABSTRACT
The seminal 2003 paper by Cosley, Lab, Albert, Konstan, and Reidl, demonstrated the susceptibility of recommender systems to rating biases. To facilitate browsing and selection, almost all recommender systems display average ratings before accepting ratings from users which has been shown to bias ratings. This effect is called Social Inuence Bias (SIB); the tendency to conform to the perceived \norm" in a community. We propose a methodology to 1) learn, 2) analyze, and 3) mitigate the effect of SIB in recommender systems. In the Learning phase, we build a baseline dataset by allowing users to rate twice: before and after seeing the average rating. In the Analysis phase, we apply a new non-parametric significance test based on the Wilcoxon statistic to test whether the data is consistent with SIB. If significant, we propose a Mitigation phase using polynomial regression and the Bayesian Information Criterion (BIC) to predict unbiased ratings. We evaluate our approach on a dataset of 9390 ratings from the California Report Card (CRC), a rating-based system designed to encourage political engagement. We found statistically significant evidence of SIB. Mitigating models were able to predict changed ratings with a normalized RMSE of 12.8% and reduce bias by 76.3%. The CRC, our data, and experimental code are available at: http://californiareportcard.org/data/
Supplemental Material
- J. Albors, J. C. Ramos, and J. L. Hervas. New learning network paradigms: Communities of objectives, crowdsourcing, wikis and open source. International Journal of Information Management, 28(3):194--202, 2008. Google Scholar
Digital Library
- X. Amatriain, J. M. Pujol, and N. Oliver. I like it... i like it not: Evaluating user ratings noise in recommender systems. In User Modeling, Adaptation, and Personalization, pages 247--258. Springer, 2009. Google ScholarDigital Library
- X. Amatriain, J. M. Pujol, N. Tintarev, and N. Oliver. Rate it again: increasing recommendation accuracy by user fire-rating. In Proceedings of the third ACM conference on Recommender systems, pages 173--180. ACM, 2009. Google ScholarDigital Library
- S. E. Asch. Opinions and social pressure. Readings about the social animal, pages 17--26, 1955.Google Scholar
- S. E. Asch. Studies of independence and conformity. American Psychological Association, 1956.Google Scholar
- A. V. Banerjee. A simple model of herd behavior. The Quarterly Journal of Economics, 107(3):797--817, 1992.Google ScholarCross Ref
- M. Bilgic and R. J. Mooney. Explaining recommendations: Satisfaction vs. promotion. In Beyond Personalization Workshop, IUI, volume 5, 2005.Google Scholar
- E. Bitton. A spatial model for collaborative filtering of comments in an online discussion forum. In Proceedings of the third ACM conference on Recommender systems, pages 393--396. ACM, 2009. Google ScholarDigital Library
- R. Bond and P. B. Smith. Culture and conformity: A meta-analysis of studies using asch's (1952b, 1956) line judgment task. Psychological bulletin, 119(1):111, 1996.Google ScholarCross Ref
- K. P. Burnham and D. R. Anderson. Model selection and multimodel inference: a practical information-theoretic approach. Springer, 2002.Google Scholar
- R. E. Burnkrant and A. Cousineau. Informational and normative social inuence in buyer behavior. Journal of Consumer research, pages 206--215, 1975.Google Scholar
- D. Cosley, S. K. Lam, I. Albert, J. A. Konstan, and J. Riedl. Is seeing believing?: how recommender system interfaces affect users' opinions. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 585--592. ACM, 2003. Google ScholarDigital Library
- C. Danescu-Niculescu-Mizil, G. Kossinets, J. Kleinberg, and L. Lee. How opinions are received by online communities: a case study on amazon. com helpfulness votes. In Proceedings of the 18th international conference on World wide web, pages 141--150. ACM, 2009. Google ScholarDigital Library
- S. DellaVigna and M. Gentzkow. Persuasion: empirical evidence. Technical report, National Bureau of Economic Research, 2009.Google Scholar
- P. M. DeMarzo, D. Vayanos, and J. Zwiebel. Persuasion bias, social inuence, and unidimensional opinions. The Quarterly Journal of Economics, 118(3):909--968, 2003.Google ScholarCross Ref
- U. M. Dholakia, S. Basuroy, and K. Soltysinski. Auction or agent (or both)? a study of moderators of the herding bias in digital auctions. International Journal of Research in Marketing, 19(2):115--130, 2002.Google ScholarCross Ref
- S. Faridani. Using canonical correlation analysis for generalized sentiment analysis, product recommendation and search. In Proceedings of the fifth ACM conference on Recommender systems, pages 355--358. ACM, 2011. Google ScholarDigital Library
- S. Faridani, E. Bitton, K. Ryokai, and K. Goldberg. Opinion space: a scalable tool for browsing online comments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1175--1184. ACM, 2010. Google ScholarDigital Library
- K. Goldberg, T. Roeder, D. Gupta, and C. Perkins. Eigentaste: A constant time collaborative filtering algorithm. Information Retrieval, 4(2):133--151, 2001. Google ScholarDigital Library
- B. Golub and M. O. Jackson. Naive learning in social networks and the wisdom of crowds. American Economic Journal: Microeconomics, pages 112--149, 2010.Google ScholarCross Ref
- H. Hong, J. D. Kubik, and J. C. Stein. Social interaction and stock-market participation. The journal of finance, 59(1):137--163, 2004.Google Scholar
- J.-H. Huang and Y.-F. Chen. Herding in online product choice. Psychology & Marketing, 23(5):413--428, 2006.Google ScholarCross Ref
- L. Jian, J. MacKie-Mason, B. Chiao, A. Levchenko, A. Zellner, J. Kmenta, J. Dreze, and W. Oberhofer. Incentive-centered design for user-contributed content. The Oxford Handbook of the Digital Economy, Oxford University Press Oxford, pages 399--433, 2012.Google Scholar
- E. L. Lehmann and H. J. D'Abrera. Nonparametrics: statistical methods based on ranks. Springer New York, 2006.Google Scholar
- J. Lorenz, H. Rauhut, F. Schweitzer, and D. Helbing. How social inuence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences, 108(22):9020--9025, 2011.Google ScholarCross Ref
- C. A. Markowski and E. P. Markowski. Conditions for the effectiveness of a preliminary test of variance. The American Statistician, 44(4):322--326, 1990.Google Scholar
- S. Moscovici and C. Faucheux. Social inuence, conformity bias, and the study of active minorities. Advances in experimental social psychology, 6:149--202, 1972.Google Scholar
- L. Muchnik, S. Aral, and S. J. Taylor. Social inuence bias: A randomized experiment. Science, 341(6146):647--651, 2013.Google ScholarCross Ref
- T. Nathanson, E. Bitton, and K. Goldberg. Eigentaste 5.0: constant-time adaptability in a recommender system using item clustering. In Proceedings of the 2007 ACM conference on Recommender systems, pages 149--152. ACM, 2007. Google ScholarDigital Library
- B. S. Noveck. Wiki-government. Democracy: A Journal of Ideas (7), 2008.Google Scholar
- K. O'Hara. Transparency, open data and trust in government: Shaping the infosphere. In Proceedings of the 3rd Annual ACM Web Science Conference, pages 223--232. ACM, 2012. Google ScholarDigital Library
- G. Schwarz et al. Estimating the dimension of a model. The annals of statistics, 6(2):461--464, 1978.Google Scholar
- S. Sharma and S. Bikhchandani. Herd behavior in financial markets-a review. International Monetary Fund, 2000.Google ScholarCross Ref
- R. Sipos, A. Ghosh, and T. Joachims. Was this review helpful to you?: it depends! context and voting patterns in online content. In Proceedings of the 23rd international conference on World wide web, pages 337--348. International World Wide Web Conferences Steering Committee, 2014. Google ScholarDigital Library
- N. Tintarev and J. Mastho. A survey of explanations in recommender systems. In Data Engineering Workshop, 2007 IEEE 23rd International Conference on, pages 801--810. IEEE, 2007. Google ScholarDigital Library
- W. Wood. Attitude change: Persuasion and social inuence. Annual review of psychology, 51(1):539--570, 2000.Google Scholar
- H. Zhu, B. Huberman, and Y. Luon. To switch or not to switch: understanding social inuence in online choices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2257--2266. ACM, 2012. Google ScholarDigital Library
Index Terms
- A methodology for learning, analyzing, and mitigating social influence bias in recommender systems
Recommendations
Mitigating Sentiment Bias for Recommender Systems
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information RetrievalBiases and de-biasing in recommender systems (RS) have become a research hotspot recently. This paper reveals an unexplored type of bias, i.e., sentiment bias. Through an empirical study, we find that many RS models provide more accurate recommendations ...
Effects of Personalized and Aggregate Top-N Recommendation Lists on User Preference Ratings
Prior research has shown a robust effect of personalized product recommendations on user preference judgments for items. Specifically, the display of system-predicted preference ratings as item recommendations has been shown in multiple studies to bias ...
A Hybrid Model Based on the Rating Bias and Textual Bias for Recommender Systems
Neural Information ProcessingAbstractMatrix Factorization is a useful approach in recommender systems. However, it only considers the user-item matrix, which will result in the data sparsity problem. To remit this issue, most researchers focus on using the item side-information to ...
Comments