Abstract
Feature critiquing has emerged as an important feedback strategy for conversational recommender systems as it offers a useful balance between user effort and recommendation efficiency. Dynamic critiquing has recently been presented as an extension to conventional (single-feature) critiquing that supports the simultaneous critiquing of multiple features. To date, dynamic critiquing has been evaluated through a variety of artificial user trials to demonstrate its potential advantages, when it comes to improving recommendation efficiency and quality. However these advantages have never been verified through any large-scale user trial. The contribution of this paper is that we present the results of such an evaluation, which confirms the advantages of dynamic critiquing in a realistic online, e-commerce setting. Furthermore we investigate the impact of implicitly maintaining session specific user models to influence the selection of compound critiques. These models are incrementally constructed as the user critiques example recommendations from cycle to cycle. Our live-user evaluation also enabled us to analyse how real users interact with the compound critiques that are produced in this way. The results demonstrate that our incremental critiquing approach has the capability of generating more relevant critique options, and that users frequently recognise the benefits associated with using these as feedback options, leading to significantly shorter recommendation sessions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., Inkeri Verkamo, A.: Fast Discovery of Association Rules in Large Databases. In: Advances in Knowledge Discovery and Data Mining, pp. 307–328 (1996)
Burke, R., Hammond, K., Young, B.: Knowledge-based Navigation of Complex Information Spaces. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 462–468. AAAI Press/MIT Press, Portland (1996)
Burke, R., Hammond, K., Young, B.C.: The FindMe Approach to Assisted Browsing. Journal of IEEE Expert 12(4), 32–40 (1997)
Faltings, B., Pu, P., Torrens, M., Viappiani, P.: Design Example-Critiquing Interaction. In: Proceedings of the International Conference on Intelligent User Interface(IUI 2004), Funchal, Madeira, Portugal, pp. 22–29. ACM Press, New York (2004)
Linden, G., Hanks, S., Lesh, N.: Interactive Assessment of User Preference Models: The Automated Travel Assistant. In: Paris, C., Jameson, A., Tasso, C. (eds.) User Modeling: Proceedings of the Sixth International Conference, pp. 67–78. Springer, Wien (1997)
McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: On the Dynamic Generation of Compound Critiques in Conversational Recommender Systems. In: De Bra, P. (ed.) Proceedings of the Third International Conference on Adaptive Hypermedia and Web-Based Systems (AH 2004), pp. 176–184. Springer, Eindhoven (2004)
McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: On the Evaluation of Dynamic Critiquing: A Large-Scale User Study. Submitted to the Twentieth National Conference on Artificial Intelligence (AAAI 2005), Pittsburgh, Pennsylvania, USA (2005)
McSherry, D.: Minimizing Dialog Length in Interactive Case-Based Reasoning. In: Nebel, B. (ed.) Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 993–998. Morgan Kaufmann, Seattle (2001)
McSherry, D.: Similarity and Compromise. In: Ashley, K.D., Bridge, D.G. (eds.) ICCBR 2003. LNCS, vol. 2689, pp. 291–305. Springer, Heidelberg (2003)
Nguyen, Q.N., Ricci, F., Cavada, D.: User Preferences Initialization and Integration in Critique-Based Mobile Recommender Systems. In: Proceedings of Artificial Intelligence in Mobile Systems 2004, in conjunction with UbiComp 2004, pp. 71–78. Iniversitat des Saarlandes Press, Nottingham (2004)
Pu, P., Kumar, P.: Evaluating Example-based Search Tools. In: Proceedings of the ACM Conference on Electronic Commerce (EC 2004), pp. 208–217. ACM Press, New York (2004)
Reilly, J., McCarthy, K., McGinty, L., Smyth, B.: Dynamic Critiquing. In: Funk, P., González Calero, P.A. (eds.) ECCBR 2004. LNCS (LNAI), vol. 3155, pp. 763–777. Springer, Heidelberg (2004)
Reilly, J., McCarthy, K., McGinty, L., Smyth, B.: Incremental Critiquing. In: Bramer, M., Coenen, F., Allen, T. (eds.) Research and Development in Intelligent Systems XXI. Proceedings of AI 2004, pp. 101–114. Springer, Cambridge (2004)
Sherin, S., Lieberman, H.: Intelligent Profiling by Example. In: Proceedings of the International Conference on Intelligent User Interfaces (IUI 2001), pp. 145–152. ACM Press, Santa Fe (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
McCarthy, K., McGinty, L., Smyth, B., Reilly, J. (2005). A Live-User Evaluation of Incremental Dynamic Critiquing. In: Muñoz-Ávila, H., Ricci, F. (eds) Case-Based Reasoning Research and Development. ICCBR 2005. Lecture Notes in Computer Science(), vol 3620. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11536406_27
Download citation
DOI: https://doi.org/10.1007/11536406_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28174-0
Online ISBN: 978-3-540-31855-2
eBook Packages: Computer ScienceComputer Science (R0)