Zhicheng Dou
Ruihua Song
ABSTRACT
Although personalized search has been proposed for many years and many personalization strategies have been investigated, it is still unclear whether personalization is consistently effective on different queries for different users, and under different search contexts. In this paper, we study this problem and get some preliminary conclusions. We present a large-scale evaluation framework for personalized search based on query logs, and then evaluate five personalized search strategies (including two click-based and three profile-based ones) using 12-day MSN query logs. By analyzing the results, we reveal that personalized search has significant improvement over common web search on some queries but it also has little effect on other queries (e.g., queries with small click entropy). It even harms search accuracy under some situations. Furthermore, we show that straightforward click-based personalization strategies perform consistently and considerably well, while profile-based ones are unstable in our experiments. We also reveal that both long-term and short-term contexts are very important in improving search performance for profile-based personalized search strategies.
- S. M. Beitzel, E. C. Jensen, A. Chowdhury, D. Grossman, and O. Frieder. Hourly analysis of a very large topically categorized web query log. In Proceedings of SIGIR '04, pages 321--328, 2004. Google Scholar
Digital Library
- J. Boyan, D. Freitag, and T. Joachims. Evaluating retrieval performance using clickthrough data. In Proceedings of AAAI Workshop on Internet Based Information Systems, 1996.Google Scholar
- A. Broder. A taxonomy of web search. SIGIR Forum, 36(2):3--10, 2002. Google ScholarDigital Library
- J. M. Carroll and M. B. Rosson. Paradox of the active user. Interfacing thought: cognitive aspects of human-computer interaction, pages 80--111, 1987. Google ScholarDigital Library
- P. A. Chirita, C. Firan, and W. Nejdl. Summarizing local context to personalize global web search. In Proceedings of CIKM '06, 2006. Google ScholarDigital Library
- P. A. Chirita, W. Nejdl, R. Paiu, and C. Kohlschutter. Using odp metadata to personalize search. In Proceedings of SIGIR '05, pages 178--185, 2005. Google ScholarDigital Library
- S. Cronen-Townsend and W. B. Croft. Quantifying query ambiguity. In Proceedings of HLT '02, pages 94--98, 2002. Google ScholarDigital Library
- C. Dwork, R. Kumar, M. Naor, and D. Sivakumar. Rank aggregation methods for the web. In Proceedings of WWW '01, pages 613--622, 2001. Google ScholarDigital Library
- P. Ferragina and A. Gulli. A personalized search engine based on web-snippet hierarchical clustering. In WWW '05: Special interest tracks and posters of the 14th international conference on World Wide Web, pages 801--810, 2005. Google ScholarDigital Library
- T. H. Haveliwala. Topic-sensitive pagerank. In Proceedings of WWW '02, 2002. Google ScholarDigital Library
- B. J. Jansen, A. Spink, J. Bateman, and T. Saracevic. Real life information retrieval: a study of user queries on the web. SIGIR Forum, 32(1):5--17, 1998. Google ScholarDigital Library
- B. J. Jansen, A. Spink, and T. Saracevic. Real life, real users, and real needs: a study and analysis of user queries on the web. Information Processing and Management, 36(2):207--227, 2000. Google ScholarDigital Library
- J. C. Borda. Mémoire sur les élections au scrution. Histoire de l'Académie Royal des Sciences, 1781.Google Scholar
- G. Jeh and J. Widom. Scaling personalized web search. In Proceedings of WWW' 03, pages 271--279,2003. Google ScholarDigital Library
- D. H. John S. Breese and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of UAI '98, pages 43--52, 1998. Google ScholarDigital Library
- R. Krovetz and W. B. Croft. Lexical ambiguity and information retrieval. Information Systems, 10(2):115--141, 1992. Google ScholarDigital Library
- U. Lee, Z. Liu, and J. Cho. Automatic identification of user goals in web search. In Proceedings of WWW '05, pages 391--400, 2005. Google ScholarDigital Library
- Y. Li, Z. Zheng, and H. K. Dai. Kdd cup-2005 report: facing a great challenge. SIGKDD Explor. Newsl.,7(2):91--99, 2005. Google ScholarDigital Library
- F. Liu, C. Yu, and W. Meng. Personalized web search by mapping user queries to categories. In Proceedings of CIKM '02, pages 558--565, 2002. Google ScholarDigital Library
- L. Page, S. Brin, R. Motwani,, and T. Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Computer Science Department, Stanford University, 1998.Google Scholar
- J. Pitkow, H. Schutze, T. Cass, R. Cooley, D. Turnbull, A. Edmonds, E. Adar, and T. Breuel. Personalized search. Commun. ACM, 45(9):50--55, 2002. Google ScholarDigital Library
- A. Pretschner and S. Gauch. Ontology based personalized search. In Proceedings of ICTAI '99, pages 391--398, 1999. Google ScholarDigital Library
- F. Qiu and J. Cho. Automatic identification of user interest for personalized search. In Proceedings of WWW '06, pages 727--736, 2006. Google ScholarDigital Library
- D. Shen, R. Pan, J. -T. Sun, J. J. Pan, K. Wu, J. Yin, and Q. Yang. Q2cυust: our winning solution to query classification in kddcup 2005. SIGKDD Explor. Newsl., 7(2):100--110, 2005. Google ScholarDigital Library
- X. Shen, B. Tan, and C. Zhai. Context-sensitive information retrieval using implicit feedback. In Proceedings of SIGIR '05, pages 43--50, 2005. Google ScholarDigital Library
- X. Shen, B. Tan, and C. Zhai. Implicit user modeling for personalized search. In Proceedings of CIKM '05, pages 824--831, 2005. Google ScholarDigital Library
- C. Silverstein, H. Marais, M. Henzinger, and M. Moricz. Analysis of a very large web search engine query log. SIGIR Forum, 33(1):6--12, 1999. Google ScholarDigital Library
- M. Speretta and S. Gauch. Personalized search based on user search histories. In Proceedings of WI '05, pages 622--628, 2005. Google ScholarDigital Library
- K. Sugiyama, K. Hatano, and M. Yoshikawa. Adaptive web search based on user profile constructed without any effort from users. In Proceedings of WWW '04, pages 675--684, 2004. Google ScholarDigital Library
- J. T. Sun, H. J. Zeng, H. Liu, Y. Lu, and Z. Chen. Cubesvd: a novel approach to personalized web search. In Proceedings of WWW '05, pages 382--390, 2005. Google ScholarDigital Library
- B. Tan, X. Shen, and C. Zhai. Mining long-term search history to improve search accuracy. In Proceedings of KDD '06, pages 718--723, 2006. Google ScholarDigital Library
- F. Tanudjaja and L. Mui. Persona: A contextualized and personalized web search. In Proceedings of HICSS'02, pages volume3, pp. 53, 2002. Google ScholarDigital Library
- J. Teevan, E. Adar, R. Jones, and M. Potts. History repeats itself: repeat queries in yahoo's logs. In Proceedings of SIGIR '06, pages 703--704, 2006. Google ScholarDigital Library
- J. Teevan, S. T. Dumais, and E. Horvitz. Beyond the commons: Investigating the value of personalizing web search. In Proceedings of PIA '05, 2005.Google Scholar
- J. Teevan, S. T. Dumais, and E. Horvitz. Personalizing search via automated analysis of interests and activities. In Proceedings of SIGIR '05, pages 449--456, 2005. Google ScholarDigital Library
- S. Wedig and O. Madani. A large-scale analysis of query logs for assessing personalization opportunities. In Proceedings of KDD '06, pages 742--747, 2006. Google ScholarDigital Library
- Y. Xie and D. R. O'Hallaron. Locality in search engine queries and its implications for caching. In INFOCOM '02, 2002.Google Scholar
Index Terms
- A large-scale evaluation and analysis of personalized search strategies
Recommendations
Improving search personalisation with dynamic group formation
SIGIR '14: Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrievalRecent research has shown that the performance of search engines can be improved by enriching a user's personal profile with information about other users with shared interests. In the existing approaches, groups of similar users are often statically ...
Personalization of web-search using short-term browsing context
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge ManagementSearch and browsing activity is known to be a valuable source of information about user's search intent. It is extensively utilized by most of modern search engines to improve ranking by constructing certain ranking features as well as by personalizing ...
Improving Ranking Consistency for Web Search by Leveraging a Knowledge Base and Search Logs
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge ManagementIn this paper, we propose a new idea called ranking consistency in web search. Relevance ranking is one of the biggest problems in creating an effective web search system. Given some queries with similar search intents, conventional approaches typically ...
Comments