Bart P. Knijnenburg
ABSTRACT
There is an increasing consensus in the field of recommender systems that we should move beyond the offline evaluation of algorithms towards a more user-centric approach. This tutorial teaches the essential skills involved in conducting user experiments, the scientific approach to user-centric evaluation. Such experiments are essential in uncovering how and why the user experience of recommender systems comes about.
- Adomavicius, G. and Tuzhilin, A. 2011. Context-Aware Recommender Systems. Recommender Systems Handbook. F. Ricci, L. Rokach, B. Shapira, and P.B. Kantor, eds. Springer US. 217--253. Google Scholar
Digital Library
- Bollen, D., Knijnenburg, B.P., Willemsen, M.C. and Graus, M. 2010. Understanding choice overload in recommender systems. Proceedings of the fourth ACM conference on Recommender systems (Barcelona, Spain, 2010), 63--70. Google ScholarDigital Library
- Knijnenburg, B.P., Reijmer, N.J.M. and Willemsen, M.C. 2011. Each to his own: how different users call for different interaction methods in recommender systems. Proceedings of the fifth ACM conference on Recommender systems (Chicago, IL, 2011), 141--148. Google ScholarDigital Library
- Knijnenburg, B.P., Schmidt-Thieme, L. and Bollen, D.G.F.M. 2010. Workshop on user-centric evaluation of recommender systems and their interfaces. Proceedings of the fourth ACM conference on Recommender systems (New York, NY, USA, 2010), 383--384. Google ScholarDigital Library
- Knijnenburg, B.P. and Willemsen, M.C. 2009. Understanding the effect of adaptive preference elicitation methods on user satisfaction of a recommender system. Proceedings of the third ACM conference on Recommender systems (New York, NY, 2009), 381--384. Google ScholarDigital Library
- Knijnenburg, B.P., Willemsen, M.C., Gantner, Z., Soncu, H. and Newell, C. 2012. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction. 22, 4--5 (2012), 441--504. Google ScholarDigital Library
- Knijnenburg, B.P., Willemsen, M.C. and Kobsa, A. 2011. A pragmatic procedure to support the user-centric evaluation of recommender systems. Proceedings of the fifth ACM conference on Recommender systems (New York, NY, USA, 2011), 321--324. Google ScholarDigital Library
- Konstan, J. and Riedl, J. 2012. Recommender systems: from algorithms to user experience. User Modeling and User-Adapted Interaction. 22, 1 (2012), 101--123. Google ScholarDigital Library
- McNee, S.M., Riedl, J. and Konstan, J.A. 2006. Being accurate is not enough. CHI '06 extended abstracts on Human factors in computing systems (Montreal, Quebec, Canada, 2006), 1097--1101. Google ScholarDigital Library
- Pu, P., Chen, L. and Hu, R. 2012. Evaluating recommender systems from the user's perspective: survey of the state of the art. User Modeling and User-Adapted Interaction. 22, 4 (2012), 317--355. Google ScholarDigital Library
- Willemsen, M., Bollen, D. and Ekstrand, M. 2011. UCERSTI 2: second workshop on user-centric evaluation of recom-mender systems and their interfaces. Proceedings of the fifth ACM conference on Recommender systems (New York, NY, USA, 2011), 395--396. Google ScholarDigital Library
Index Terms
- Conducting user experiments in recommender systems
Recommendations
A pragmatic procedure to support the user-centric evaluation of recommender systems
RecSys '11: Proceedings of the fifth ACM conference on Recommender systemsAs recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also "online" with test users to ensure good user experience. The user evaluation of recommenders is however complex ...
Evaluating Intelligent User Interfaces with User Experiments
IUI '16 Companion: Companion Publication of the 21st International Conference on Intelligent User InterfacesUser experiments are an essential tool to evaluate the user experience of intelligent user interfaces. This tutorial teaches the practical aspects of designing and setting up user experiments, as well as state-of-the-art methods to statistically ...
CRS-Que: A User-centric Evaluation Framework for Conversational Recommender Systems
An increasing number of recommendation systems try to enhance the overall user experience by incorporating conversational interaction. However, evaluating conversational recommender systems (CRSs) from the user’s perspective remains elusive. The GUI-based ...
Comments