Tien T. Nguyen
Michael D. Ekstrand
Martijn C. Willemsen
John Riedl
ABSTRACT
One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to evaluate the efficacy of the new interfaces via a user experiment. Our results suggest that presenting exemplars during the rating process helps users rate more consistently, and increases the quality of the data.
- G. Adomavicius, J. Bockstedt, S. Curley, and J. Zhang. Recommender systems, consumer preferences, and anchoring effects. InDecisions@RecSys Workshop, pages 35--42, Chicago, 2011.Google Scholar
- X. Amatriain, J. M. Pujol, N. Tintarev, and N. Oliver. Rate it again: increasing recommendation accuracy by user re-rating. In Proc. of RecSys '09, pages 173--180, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- J. R. Bettman, M. F. Luce, and J. W. Payne. Constructive consumer choice processes. Journal of Consumer Research, 25(3):187 -- 217, Dec. 1998. ArticleType: research-article /Full publication date: December 1998 / Copyright 1998 Journal of Consumer Research Inc.Google ScholarCross Ref
- D. Bollen, M. Graus, and M. C. Willemsen. Remembering the stars': effect of time on preference retrieval from memory. In Proc. of RecSys '12, pages 217 -- 220, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- D. Cosley, S. K. Lam, I. Albert, J. A. Konstan, and J. Riedl. Is seeing believing?: how recommender system interfaces affect users' opinions. In In Proc. of CHI '03, pages 585--592, New York, NY, USA, 2003. ACM. Google ScholarDigital Library
- M. D. Ekstrand, M. Ludwig, J. A. Konstan, and J. T. Riedl. Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit. In In Proc. of RecSys '11, pages 133 -- 140, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- B. Fischho. Value elicitation: Is there anything in there? American Psychologist, 46(8):835--847, 1991.Google ScholarCross Ref
- F. M. Harper, X. Li, Y. Chen, and J. A. Konstan. An economic model of user rating in an online recommender 7 http://www.net ixprize.com system. In User Modeling 2005, pages 307--316. Springer, 2005. Google ScholarDigital Library
- J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1):5--53, 2010. Google ScholarDigital Library
- W. Hill, L. Stead, M. Rosenstein, and G. Furnas. Recommending and evaluating choices in a virtual community of use. In In Proc. of CHI '95, pages 194--201, New York, NY, USA, 1995. Google ScholarDigital Library
- C. K. Hsee. The evaluability hypothesis: An explanation for preference reversals between joint and separate evaluations of alternatives. Organizational Behavior and Human Decision Processes, 67(3):247--257, Sept. 1996.Google ScholarCross Ref
- L.-t. Hu and P. Bentler. Cuto criteria for t indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1):1--55, 1999.Google ScholarCross Ref
- D. Kluver, T. T. Nguyen, M. Ekstrand, S. Sen, and J. Riedl. How many bits per rating? In In Proc. of RecSys '12, pages 99 -- 106, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- B. P. Knijnenburg, M. C. Willemsen, Z. Gantner, H. Soncu, and C. Newell. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22(4-5):441--504, Mar. 2012. Google ScholarDigital Library
- S. Lichtenstein and P. Slovic. The Construction of Preference. Cambridge University Press, Sept. 2006.Google ScholarCross Ref
- S. M. McNee, S. K. Lam, J. A. Konstan, and J. Riedl. Interfaces for eliciting new user preferences in recommender systems. In P. Brusilovsky, A. Corbett, and F. d. Rosis, editors, User Modeling 2003, number 2702 in Lecture Notes in Computer Science, pages 178--187. Springer Berlin Heidelberg, Jan. 2003. Google ScholarDigital Library
- T. Mussweiler. Comparison processes in social judgment: Mechanisms and consequences. Psychological Review, 110(3):472--489, 2003.Google ScholarCross Ref
- T. T. Nguyen and J. Riedl. Predicting users' preference from tag relevance. User Modeling, Adaptation, and Personalization, UMAP 2013, pages 274--280, 2013.Google Scholar
- S. Nobarany, L. Oram, V. K. Rajendran, C.-H. Chen, J. McGrenere, and T. Munzner. The design space of opinion measurement interfaces: exploring recall support for rating and ranking. In Proc. of CHI '12, pages 2035 -- 2044, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- M. P. O'Mahony, N. J. Hurley, and G. C. Silvestre. Detecting noise in recommender system databases. In Proceedings of the 11th international conference on Intelligent user interfaces, IUI '06, pages 109 -- 115, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- A. Said, B. J. Jain, S. Narr, and T. Plumbaum. Users and noise: The magic barrier of recommender systems. In UMAP 2012, pages 237--248. Springer, 2012. Google ScholarDigital Library
- I. Simonson. Determinants of customers' responses to customized offers: Conceptual framework and research propositions. Journal of Marketing, 69(1):32--45, Jan. 2005.Google ScholarCross Ref
- E. I. Sparling and S. Sen. Rating: how difficult is it? In Proc. of RecSys '11, pages 149 -- 156, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- A. Tversky and D. Kahneman. Judgment under uncertainty: Heuristics and biases. Science, 185(4157):1124--1131, Sept. 1974.Google ScholarCross Ref
- A. Tversky, S. Sattath, and P. Slovic. Contingent weighting in judgment and choice. Psychological Review, 95(3):371--384, 1988.Google ScholarCross Ref
- J. Vig, S. Sen, and J. Riedl. The tag genome: Encoding community knowledge to support novel interaction. ACM Trans. Interact. Intell. Syst., 2(3):13:1 -- 13:44, Sept. 2012. Google ScholarDigital Library
- E. U. Weber and E. J. Johnson. Mindful judgment and decision making. In Annual Review of Psychology, volume 60, pages 53--85. Annual Reviews, Palo Alto, 2009.Google Scholar
Index Terms
- Rating support interfaces to improve user experience and recommender accuracy
Recommendations
User Personality and User Satisfaction with Recommender Systems
In this study, we show that individual users' preferences for the level of diversity, popularity, and serendipity in recommendation lists cannot be inferred from their ratings alone. We demonstrate that we can extract strong signals about individual ...
A recursive prediction algorithm for collaborative filtering recommender systems
RecSys '07: Proceedings of the 2007 ACM conference on Recommender systemsCollaborative filtering (CF) is a successful approach for building online recommender systems. The fundamental process of the CF approach is to predict how a user would like to rate a given item based on the ratings of some nearest-neighbor users (user-...
Expectation, Perception, and Accuracy in News Recommender Systems: Understanding the Relationships of User Evaluation Criteria Using Direct Feedback
HCI International 2021 - Late Breaking Papers: Design and User ExperienceAbstractTypically, a user-focused approach of evaluation of recommender systems requires the users to recollect their experiences, exposing study results to memory biases. In this paper, we describe a study conducted to test a framework, that allows ...
Comments