Jiaxin Mao
Yiqun Liu
Noriko Kando
Min Zhang
Skip Abstract Section
Abstract
People often conduct exploratory search to explore unfamiliar information space and learn new knowledge. While supporting the highly dynamic and interactive exploratory search is still challenging for the search system, we want to investigate which factors can make the exploratory search successful and satisfying from the user’s perspective. Previous research suggests that domain experts have different search strategies and are more successful in finding domain-specific information, but how the domain expertise level will influence users’ interaction and search outcomes in exploratory search, especially in different knowledge domains, is still unclear. In this work, via a carefully designed user study that involves 30 participants, we investigate the influence of domain expertise levels on the interaction and outcome of exploratory search in three different domains: environment, medicine, and politics. We record participants’ search behaviors, including their explicit feedback and eye fixation sequences, in a laboratory setting. With this dataset, we identify both domain-independent and domain-dependent effects on user behaviors and search outcomes. Our results extend existing research on the effect of domain expertise in search and suggest different strategies for exploiting domain expertise to support exploratory search in different knowledge domains.
- John R. Anderson. 2013. The Architecture of Cognition. Psychology Press.Google Scholar
- Kumaripaba Athukorala, Dorota Głowacka, Giulio Jacucci, Antti Oulasvirta, and Jilles Vreeken. 2016. Is exploratory search different? A comparison of information search behavior for exploratory and lookup tasks. Journal of the Association for Information Science and Technology 67, 11 (2016), 2635--2651. Google ScholarDigital Library
- Kumaripaba Athukorala, Antti Oulasvirta, Dorota Głowacka, Jilles Vreeken, and Giulio Jacucci. 2014. Narrow or broad?: Estimating subjective specificity in exploratory search. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM’14). ACM, 819--828. Google ScholarDigital Library
- Suresh K. Bhavnani. 2001. Important cognitive components of domain-specific search knowledge. In Proceedings of the Tenth Text Retrieval Conference (TREC’01). 571--578.Google Scholar
- Suresh K. Bhavnani. 2002. Domain-specific search strategies for the effective retrieval of healthcare and shopping information. In CHI’02 Extended Abstracts on Human Factors in Computing Systems. ACM, 610--611. Google ScholarDigital Library
- Pia Borlund. 2000. Experimental components for the evaluation of interactive information retrieval systems. Journal of Documentation 56, 1 (2000), 71--90.Google ScholarCross Ref
- Georg Buscher, Andreas Dengel, and Ludger van Elst. 2008. Eye movements as implicit relevance feedback. In CHI’08 Extended Abstracts on Human Factors in Computing Systems. ACM, 2991--2996. Google ScholarDigital Library
- Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Hilsdale, NJ: Lawrence Earlbaum Associates.Google Scholar
- Michael J. Cole, Jacek Gwizdka, Chang Liu, Nicholas J. Belkin, and Xiangmin Zhang. 2013. Inferring user knowledge level from eye movement patterns. Information Processing 8 Management 49, 5 (2013), 1075--1091. Google ScholarDigital Library
- Michael J. Cole, Xiangmin Zhang, Chang Liu, Nicholas J. Belkin, and Jacek Gwizdka. 2011. Knowledge effects on document selection in search results pages. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’11). ACM, 1219--1220. Google ScholarDigital Library
- Kevyn Collins-Thompson, Soo Young Rieh, Carl C. Haynes, and Rohail Syed. 2016. Assessing learning outcomes in web search: A comparison of tasks and query strategies. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR’16). ACM, 163--172. Google ScholarDigital Library
- Geoffrey B. Duggan and Stephen J. Payne. 2008. Knowledge in the head and on the web: Using topic expertise to aid search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’08). ACM, 39--48. Google ScholarDigital Library
- Yuka Egusa, Hitomi Saito, Masao Takaku, Hitoshi Terai, Makiko Miwa, and Noriko Kando. 2010. Using a concept map to evaluate exploratory search. In Proceedings of the Third Symposium on Information Interaction in Context (IIiX’10). ACM, 175--184. Google ScholarDigital Library
- Carsten Eickhoff, Sebastian Dungs, and Vu Tran. 2015. An eye-tracking study of query reformulation. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, 13--22. Google ScholarDigital Library
- Carsten Eickhoff, Jaime Teevan, Ryen White, and Susan Dumais. 2014. Lessons from the journey: A query log analysis of within-session learning. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM’14). ACM, 223--232. Google ScholarDigital Library
- Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G* power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods 39, 2 (2007), 175--191.Google ScholarCross Ref
- Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. 2005. Evaluating implicit measures to improve web search. ACM Transactions on Information Systems (TOIS) 23, 2 (2005), 147--168. Google ScholarDigital Library
- Ahmed Hassan Awadallah, Ryen W. White, Patrick Pantel, Susan T. Dumais, and Yi-Min Wang. 2014. Supporting complex search tasks. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM’14). ACM, 829--838. Google ScholarDigital Library
- Jiepu Jiang, Daqing He, and James Allan. 2014. Searching, browsing, and clicking in a search session: Changes in user behavior by task and over time. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’14). ACM, 607--616. Google ScholarDigital Library
- Ruogu Kang and Wai-Tat Fu. 2010. Exploratory information search by domain experts and novices. In Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI’10). ACM, 329--332. Google ScholarDigital Library
- Evangelos Kanoulas, Ben Carterette, Mark Hall, Paul Clough, and Mark Sanderson. 2011. Overview of the trec 2011 session track. In Proceedings of the Twentieth Text REtrieval Conference (TREC’11).Google Scholar
- Weize Kong and James Allan. 2014. Extending faceted search to the general web. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM’14). ACM, 839--848. Google ScholarDigital Library
- Tessa Lau and Eric Horvitz. 1999. Patterns of search: Analyzing and modeling web query refinement. In User Modeling (UM’99). Springer, 119--128. Google ScholarDigital Library
- Xin Li, Yiqun Liu, Rongjie Cai, and Shaoping Ma. 2017. Investigation of user search behavior while facing heterogeneous search services. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM’17). ACM, 161--170. Google ScholarDigital Library
- Yuelin Li and Nicholas J. Belkin. 2008. A faceted approach to conceptualizing tasks in information seeking. Information Processing 8 Management 44, 6 (2008), 1822--1837. Google ScholarDigital Library
- Chang Liu, Xiangmin Zhang, and Wei Huang. 2016. The exploration of objective task difficulty and domain knowledge effects on users’ query formulation. In Proceedings of the Association for Information Science and Technology (ASIST’16). American Society for Information Science, Article 63, 9 pages. Google ScholarDigital Library
- Mengyang Liu, Yiqun Liu, Jiaxin Mao, Cheng Luo, Min Zhang, and Shaoping Ma. 2018. “Satisfaction with failure” or “unsatisfied success”: Investigating the relationship between search success and user satisfaction. In Proceedings of the 2018 World Wide Web Conference (WWW’18). ACM. Google ScholarDigital Library
- Yiqun Liu, Ye Chen, Jinhui Tang, Jiashen Sun, Min Zhang, Shaoping Ma, and Xuan Zhu. 2015. Different users, different opinions: Predicting search satisfaction with mouse movement information. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, 493--502. Google ScholarDigital Library
- Zeyang Liu, Yiqun Liu, Ke Zhou, Min Zhang, and Shaoping Ma. 2015. Influence of vertical result in web search examination. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, 193--202. Google ScholarDigital Library
- Henry B. Mann and Donald R. Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics 18, 1 (1947), 50--60.Google ScholarCross Ref
- Jiaxin Mao, Yiqun Liu, Ke Zhou, Jian-Yun Nie, Jingtao Song, Min Zhang, Shaoping Ma, Jiashen Sun, and Hengliang Luo. 2016. When does relevance mean usefulness and user satisfaction in web search? In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’16). ACM, 463--472. Google ScholarDigital Library
- Gary Marchionini. 2006. Exploratory search: From finding to understanding. Communications of the ACM 49, 4 (2006), 41--46. Google ScholarDigital Library
- Daan Odijk, Ryen W. White, Ahmed Hassan Awadallah, and Susan T. Dumais. 2015. Struggling and success in web search. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM’15). ACM, 1551--1560. Google ScholarDigital Library
- Eyal M. Reingold, Erik D. Reichle, Mackenzie G. Glaholt, and Heather Sheridan. 2012. Direct lexical control of eye movements in reading: Evidence from a survival analysis of fixation durations. Cognitive Psychology 65, 2 (2012), 177--206.Google ScholarCross Ref
- Lynda Tamine and Cecile Chouquet. 2017. On the impact of domain expertise on query formulation, relevance assessment and retrieval performance in clinical settings. Information Processing 8 Management 53, 2 (2017), 332--350. Google ScholarDigital Library
- Pertti Vakkari, Mikko Pennanen, and Sami Serola. 2003. Changes of search terms and tactics while writing a research proposal: A longitudinal case study. Information Processing 8 Management 39, 3 (2003), 445--463. Google ScholarDigital Library
- Ryen W. White, Susan T. Dumais, and Jaime Teevan. 2009. Characterizing the influence of domain expertise on web search behavior. In Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM’09). ACM, 132--141. Google ScholarDigital Library
- Ryen W. White and Resa A. Roth. 2009. Exploratory search: Beyond the query-response paradigm. Synthesis Lectures on Information Concepts, Retrieval, and Services 1, 1 (2009), 1--98.Google ScholarCross Ref
- Barbara M. Wildemuth. 2004. The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science and Technology 55, 3 (2004), 246--258. Google ScholarDigital Library
- Xiangmin Zhang, Hermina G. B. Anghelescu, and Xiaojun Yuan. 2005. Domain knowledge, search behaviour, and search effectiveness of engineering and science students: An exploratory study. Information Research: An International Electronic Journal 10, 2 (2005), n2.Google ScholarCross Ref
- Xiangmin Zhang, Jingjing Liu, Michael Cole, and Nicholas Belkin. 2015. Predicting users’ domain knowledge in information retrieval using multiple regression analysis of search behaviors. Journal of the Association for Information Science and Technology 66, 5 (2015), 980--1000. Google ScholarDigital Library
Index Terms
- How Does Domain Expertise Affect Users’ Search Interaction and Outcome in Exploratory Search?
Recommendations
Characterizing the influence of domain expertise on web search behavior
WSDM '09: Proceedings of the Second ACM International Conference on Web Search and Data MiningDomain experts search differently than people with little or no domain knowledge. Previous research suggests that domain experts employ different search strategies and are more successful in finding what they are looking for than non-experts. In this ...
How medical expertise influences web search interaction
SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrievalDomain expertise can have an important influence on how people search. In this poster we present findings from a log-based study into how medical domain experts search the Web for information related to their expertise, as compared with non-experts. We ...
Exploratory information search by domain experts and novices
IUI '10: Proceedings of the 15th international conference on Intelligent user interfacesThe arising popularity of social tagging system has the potential to transform traditional web search into a new era of social search. Based on the finding that domain expertise could influence search behavior in traditional search engines, we ...
Comments