Wei Li
Justin Matejka
Tovi Grossman
Joseph A. Konstan
Abstract
We examine the use of modern recommender system technology to aid command awareness in complex software applications. We first describe our adaptation of traditional recommender system algorithms to meet the unique requirements presented by the domain of software commands. A user study showed that our item-based collaborative filtering algorithm generates 2.1 times as many good suggestions as existing techniques. Motivated by these positive results, we propose a design space framework and its associated algorithms to support both global and contextual recommendations. To evaluate the algorithms, we developed the CommunityCommands plug-in for AutoCAD. This plug-in enabled us to perform a 6-week user study of real-time, within-application command recommendations in actual working environments. We report and visualize command usage behaviors during the study, and discuss how the recommendations affected users behaviors. In particular, we found that the plug-in successfully exposed users to new commands, as unique commands issued significantly increased.
- Adomavicius, G. and Tuzhilin, A. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Engin. 17, 734--749. Google Scholar
Digital Library
- Applecomputers 2009. iTunes.Google Scholar
- Baecker, R., Booth, K., Jovicic, S., Mcgrenere, J., and Moore, G. 2000. Reducing the gap between what users know and what they need to know. In Proceedings of the Conference on Universal Usability. ACM Press, 17--23. Google ScholarDigital Library
- Bailey, B. P. and Konstan, J. A. 2006. On the need for attention-aware systems: measuring effects of interruption on task performance, error rate, and affective state. Comput. Hum. Behav. 22, 685--708.Google ScholarCross Ref
- Basu, C., Hirsh, H., and Cohen, W. 1998. Recommendation as classification: using social and content-based information in recommendation. In Proceedings of the 15th National Conference on Artificial Intelligence (AAAI'98). AAAI Press, 714--720. Google ScholarDigital Library
- Bosser, T. 1987. Learning in Man-Computer Interaction: A Review of the Literature. Springer, Berlin.Google Scholar
- Breese, J. S., Heckerman, D., and Kadie, C. 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI'98). Elsevier Science, 43--52. Google ScholarDigital Library
- Canny, J. 2002. Collaborative filtering with privacy via factor analysis. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'02). ACM Press, 238--245. Google ScholarDigital Library
- Carroll, J. M. and Carrithers, C. 1984. Training wheels in a user interface. Com. ACM 27, 800--806. Google ScholarDigital Library
- Celma, O. and Herrera, P. 2008. A new approach to evaluating novel recommendations. In Proceedings of the 2nd ACM Conference on Recommender Systems (RecSys'08). ACM Press, 179--186. Google ScholarDigital Library
- Chen, L. and Pu, P. 2006. Evaluating critiquing-based recommender agents. In Proceedings of the 21st National Conference on Artificial intelligence (AAAI'06). AAAI Press, 157--162. Google ScholarDigital Library
- Claypool, M., Le, P., Wased, M., and Brown, D. 2001. Implicit interest indicators. In Proceedings of the 6th International Conference on Intelligent User Interfaces (IUI'01). ACM Press, 33--40. Google ScholarDigital Library
- Cutrell, E., Czerwinski, M., and Horvitz, E. 2001. Notification, disruption and memory: effects of messaging interruptions on memory and performance. In Proceedings of the 8th Conference on Human-computer Interaction (INTERACT'01). IOS Press, 263--269.Google Scholar
- Cutrell, E. B., Czerwinski, M., and Horvitz, E. 2000. Effects of instant messaging interruptions on computing tasks. In Proceedings of the ACM CHI Conference on Human Factors in Computing Systems (CHI'00). ACM Press, 99--100. Google ScholarDigital Library
- Davison, B. and Hirsh, H. 1998. Predicting sequences of user actions. In Proceedings of the AAAI/ICML Whorkshop on Predicting the Future: AI Approaches to Time-Series Analysis. 5--12.Google Scholar
- Dent, L., Boticario, J., Mcdermott, J., Mitchell, T., and Zabowski, D. 1992. A personal learning apprentice. In Proceedings of the 10th National Conference on Artificial Intelligence (AAAI'92). AAAI Press, 96--103. Google ScholarDigital Library
- Ding, Y. and Li, X. 2005. Time weight collaborative filtering. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management. ACM Press, 485--492. Google ScholarDigital Library
- Ding, Y., Li, X., and Orlowska, M. E. 2006. Recency-based collaborative filtering. In Proceedings of the 17th Australasian Database Conference. Australian Computer Society, Inc., 99--107. Google ScholarDigital Library
- Faltings, B., Pu, P., Torrens, M., and Viappiani, P. 2004. Designing example-critiquing interaction. In Proceedings of the 9th International Conference on Intelligent User Interfaces (IUI'04). ACM Press, 22--29. Google ScholarDigital Library
- Farzan, R. and Brusilovsky, P. 2006. Social navigation support in a course recommendation system. In Proceedings of the 4th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems. Elsevier, 91--100. Google ScholarDigital Library
- Findlater, L. and Mcgrenere, J. 2010. Beyond performance: Feature awareness in personalized interfaces. Int. J. Hum.-Comput. Stud. 68, 121--137. Google ScholarDigital Library
- Fischer, G. 2001. User modeling in human-computer interaction. User Model. User-Adapt. Interac. 11, 65--86. Google ScholarDigital Library
- Gajos, K. Z., Weld, D. S., and Wobbrock, J. O. 2010. Automatically generating personalized user interfaces with Supple. Artif. Intell. 174, 910--950. Google ScholarDigital Library
- Grossman, T., Fitzmaurice, G., and Attar, R. 2009. A survey of software learnability: Metrics, methodologies and guidelines. In Proceedings of the ACM CHI Conference on Human Factors in Computing Systems (CHI'09). 649--658. Google ScholarDigital Library
- Herlocker, J., Konstan, J. A., and Riedl, J. 2002. An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inform. Retri. 5, 287--310. Google ScholarDigital Library
- Herlocker, J. L., Konstan, J. A., Borchers, A., and Riedl, J. 1999. An algorithmic framework for performing collaborative filtering. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'99). ACM Press, 230--237. Google ScholarDigital Library
- Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. 2004. Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22, 5--53. Google ScholarDigital Library
- Hermens, L. A. and Schlimmer, J. C. 1993. A machine-learning apprentice for the completion of repetitive forms. In Proceedings of the 9th IEEE Conference on Artificial Intelligence Applications. IEEE Computer Society Press, Los Alamitos, CA, 164--170.Google Scholar
- Hill, W., Stead, L., Rosenstein, M., and Furnas, G. 1995. Recommending and evaluating choices in a virtual community of use. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'95). ACM Press/Addison-Wesley Publishing Co., 194--201. Google ScholarDigital Library
- Hofmann, T. 2004. Latent semantic models for collaborative filtering. ACM Trans. Inf. Syst. 22, 89--115. Google ScholarDigital Library
- Horvitz, E. 1999. Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'99). ACM, 159--166. Google ScholarDigital Library
- Horvitz, E., Breese, J., Heckerman, D., Hovel, D., and Rommelse, K. 1998. The Lumière project: Bayesian user modeling for inferring the goals and needs of software users. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI'98). Morgan Kaufmann, 256--265. Google ScholarDigital Library
- Hsi, I. and Potts, C. 2000. Studying the evolution and enhancement of software features. In Proceedings of the International Conference on Software Maintenance (ICSM'00). IEEE Computer Society, 143--151. Google ScholarDigital Library
- Hsu, M.-H. 2008. A personalized English learning recommender system for ESL students. Expert Syst. Appl. 34, 683--688. Google ScholarDigital Library
- Hudson, S., Fogarty, J., Atkeson, C., Avraham, D., Forlizzi, J., Kiesler, S., Lee, J., and Yang, J. 2003. Modeling user behavior: Predicting human interruptibility with sensors: A Wizard of Oz feasibility study. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'03). 257--264. Google ScholarDigital Library
- Igarashi, T. and Hughes, J. 2001. A suggestive interface for 3D drawing. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'01). 173--181. Google ScholarDigital Library
- Jones, K. S. 1972. A statistical interpretation of specificity and its application in retrieval. J. Documenta. 60, 493--502.Google ScholarCross Ref
- Kaufman, L. and Weed, B. 1998. Too much of a good thing?: Identifying and resolving bloat in the user interface. In Proceedings of the SIGCHI Conference Summary on Human Factors in Computing Systems (CHI'98). ACM, 207--208. Google ScholarDigital Library
- Knabe, K. 1995. Apple guide: a case study in user-aided design of online help. In Proceedings of the Conference Companion on Human Factors in Computing Systems (CHI'95). ACM, 286--287. Google ScholarDigital Library
- Koren, Y. 2009. Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 447--456. Google ScholarDigital Library
- Krzywicki, A., Wobcke, W., and Wong, A. 2010. An adaptive calendar assistant using pattern mining for user preference modelling. In Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI'10). ACM, 71--80. Google ScholarDigital Library
- Lafreniere, B., Bunt, A., Whissell, J. S., Clarke, C. L. A., and Terry, M. 2010. Characterizing large-scale use of a direct manipulation application in the wild. In Proceedings of Graphics Interface (GI'10). Canadian Information Processing Society, 11--18. Google ScholarDigital Library
- Linden, G., Smith, B., and York, J. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Intern. Comput. 7, 76--80. Google ScholarDigital Library
- Linton, F. and Schaefer, H.-P. 2000. Recommender systems for learning: building user and expert models through long-term observation of application use. User Model. User-Adapt. Interact. 10, 181--208. Google ScholarDigital Library
- Liu, J., Wong, C. K., and Hui, K. K. 2003. An adaptive user interface based on personalized learning. Intell. Syst. 18, 52--57. Google ScholarDigital Library
- Mack, R. 1990. Understanding and learning text-editing skills: observations on the role of new user expectations. In Cognition, Computing, and Cooperation, Ablex Publishing Corp., 304--337. Google ScholarDigital Library
- Matejka, J., Li, W., Grossman, T., and Fitzmaurice, G. 2009. CommunityCommands: command recommendations for software applications. In Proceedings of the 22nd Symposium on User Interface Software and Technology (UIST'09). Google ScholarDigital Library
- Mccrickard, D. S., Czerwinski, M., and Bartram, L. 2003. Introduction: design and evaluation of notification user interfaces. Int. J. Hum.-Comput. Stud. 58, 509--514. Google ScholarDigital Library
- Mcgrenere, J., Baecker, R. M., and Booth, K. S. 2002. An evaluation of a multiple interface design solution for bloated software. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'02). 163--170. Google ScholarDigital Library
- Mcnee, S. M., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S. K., Rashid, A. M., Konstan, J. A., and Riedl, J. 2002. On the recommending of citations for research papers. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'02). ACM, 116--125. Google ScholarDigital Library
- Mcnee, S. M., Kapoor, N., and Konstan, J. A. 2006a. Don't look stupid: Avoiding pitfalls when recommending research papers. In Proceedings of the 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW'06). ACM, 171--180. Google ScholarDigital Library
- Mcnee, S. M., Riedl, J., and Konstan, J. A. 2006b. Being accurate is not enough: How accuracy metrics have hurt recommender systems. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1097--1101. Google ScholarDigital Library
- Miller, B. N., Albert, I., Lam, S. K., Konstan, J. A., and Riedl, J. 2003. MovieLens unplugged: experiences with an occasionally connected recommender system. In Proceedings of the 8th International Conference on Intelligent User Interfaces (IUI'03). ACM, 263--266. Google ScholarDigital Library
- Nguyen, Q. N. and Ricci, F. 2008. Long-term and session-specific user preferences in a mobile recommender system. In Proceedings of the 13th International Conference on Intelligent User Interfaces (IUI'08). ACM, 381--384. Google ScholarDigital Library
- Norman, D. A. and Draper, S. W. 1986. User Centered System Design; New Perspectives on Human-Computer Interaction. L. Erlbaum Associates Inc. Google ScholarDigital Library
- Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., and Riedl, J. 1994. GroupLens: An open architecture for collaborative filtering of netnews. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'94). ACM, 175--186. Google ScholarDigital Library
- Sarwar, B., Karypis, G., Konstan, J., and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web. ACM, 285--295. Google ScholarDigital Library
- Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. 2000. Analysis of recommendation algorithms for e-commerce. In Proceedings of the 2nd ACM Conference on Electronic Commerce. ACM, 158--167. Google ScholarDigital Library
- Schafer, J. B., Konstan, J., and Riedi, J. 1999. Recommender systems in e-commerce. In Proceedings of the 1st ACM Conference on Electronic Commerce. ACM, 158--166. Google ScholarDigital Library
- Shneiderman, B. 1983. Direct manipulation: A step beyond programming languages. Comput. 16, 57--69. Google ScholarDigital Library
- Shneiderman, B. 2003. Promoting universal usability with multi-layer interface design. In Proceedings of the Conference on Universal Usability. ACM, 1--8. Google ScholarDigital Library
- Terry, M. and Mynatt, E. D. 2002. Side views: persistent, on-demand previews for open-ended tasks. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (UIST'02). ACM, 71--80. Google ScholarDigital Library
- Terveen, L., Mcmackin, J., Amento, B., and Hill, W. 2002. Specifying preferences based on user history. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'02). ACM, 315--322. Google ScholarDigital Library
- Weng, L.-T., Xu, Y., Li, Y., and Nayak, R. 2007. Improving recommendation novelty based on topic taxonomy. In Proceedings of the IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology Workshops. IEEE Computer Society, 115--118. Google ScholarDigital Library
- Wisneski, C., Ishii, H., Dahley, A., Gorbet, M. G., Brave, S., Ullmer, B., and Yarin, P. 1998. Ambient displays: Turning architectural space into an interface between people and digital information. In Proceedings of the 1st International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture. Springer-Verlag, 22--32. Google ScholarDigital Library
- Witten, I. H., Cleary, J. G., and Greenberg, S. 1984. On frequency-based menu-splitting algorithms. Int. J. Man-Mach. Stud., 135--148.Google Scholar
- Xiao, J., Stasko, J., and Catrambone, R. 2004. An empirical study of the effect of agent competence on user performance and perception. In Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems. IEEE Computer Society, 178--185. Google ScholarDigital Library
- Zhang, Y., Callan, J., and Minka, T. 2002. Novelty and redundancy detection in adaptive filtering. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'02). ACM, 81--88. Google ScholarDigital Library
- Ziegler, C.-N., Mcnee, S. M., Konstan, J. A., and Lausen, G. 2005. Improving recommendation lists through topic diversification. In Proceedings of the 14th International Conference on World Wide Web. ACM, 22--32. Google ScholarDigital Library
- Zipf, G. K. 1949. Human Behavior and The Principle of Least Effort: An Introduction to Human Ecology. Addison-Wesley Press, Cambridge, MA.Google Scholar
Index Terms
- Design and evaluation of a command recommendation system for software applications
Recommendations
Parallel Ratio Based CF for Recommendation System
ICCCNT '16: Proceedings of the 7th International Conference on Computing Communication and Networking TechnologiesWith the increase in E-commerce, Recommendation Systems are getting popular to provide recommendations of various items (movies, books, music) to users. To build the Recommendation System (RS), Collaborative Filtering (CF) techniques are proven ...
Typicality-Based Collaborative Filtering Recommendation
Collaborative filtering (CF) is an important and popular technology for recommender systems. However, current CF methods suffer from such problems as data sparsity, recommendation inaccuracy, and big-error in predictions. In this paper, we borrow ideas ...
New Recommendation Techniques for Multicriteria Rating Systems
Traditional single-rating recommender systems have been successful in a number of personalization applications, but the research area of multicriteria recommender systems has been largely untouched. Taking full advantage of multicriteria ratings in ...
Reviews
Alexis Leon
The prime mover behind today's digital revolution is computer software. As the role of software becomes increasingly prominent, software applications are becoming more complex"?in size, sophistication, and technologies used. To stay competitive, software vendors are introducing more features in each new version of their products. In order to reduce the learning curve and make the software more usable, software developers include user manuals, online help, context-sensitive help, and tips to educate users. But the majority of users don't use these resources. Furthermore, they only use a fraction of the thousands of commands and features that are available to them. In this paper, the authors discuss the limitations of the existing recommender systems. Recommender systems help users use software applications more efficiently, and they make users aware of any new features and capabilities. The paper examines the technologies behind modern recommender systems. It explains why the authors have chosen the collaborative filtering algorithm, including how they adapted it for software command recommendations. Collaborative filtering makes automatic predictions about a user's interests by collecting preferences or taste information from many users. The authors' goal was to develop collaborative filtering algorithms that will produce recommendations that are novel (commands that are unfamiliar to the user) as well as useful (immediately or in the future). The system was designed by taking into consideration the command usage history of the user community, as well as that of the active user's current session, along with available historical data to produce command recommendations. The authors used memory-based collaborative filtering algorithms to produce the recommendations. A prototype named CommunityCommands was developed as a plug-in for AutoCAD. The authors review and discuss the results of the field study of the prototype and the importance of the groundwork they have done for deploying a real tool. They also establish the superiority of their methodology over existing technologies. This paper concludes by pointing to the different directions in which this work could be extended in order to create better recommender systems and thereby better software applications. This paper is a must-read for software application designers and developers. It will also be extremely useful to usability experts. This is an area with a lot of potential, and thus it should also interest computer science and information technology students and researchers. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments