Yun Zhou
W. Bruce Croft
ABSTRACT
Current prediction techniques, which are generally designed for content-based queries and are typically evaluated on relatively homogenous test collections of small sizes, face serious challenges in web search environments where collections are significantly more heterogeneous and different types of retrieval tasks exist. In this paper, we present three techniques to address these challenges. We focus on performance prediction for two types of queries in web search environments: content-based and Named-Page finding. Our evaluation is mainly performed on the GOV2 collection. In addition to evaluating our models for the two types of queries separately, we consider a more challenging and realistic situation that the two types of queries are mixed together without prior information on query types. To assist prediction under the mixed-query situation, a novel query classifier is adopted. Results show that our prediction of web query performance is substantially more accurate than the current state-of-the-art prediction techniques. Consequently, our paper provides a practical approach to performance prediction in real-world web settings.
- Y. Zhou, W.B. Croft, Ranking Robustness: A Novel Framework to Predict Query Performance, in Proceedings of CIKM 2006. Google Scholar
Digital Library
- D. Carmel, E. Yom-Tov, A. Darlow, D. Pelleg, What Makes a Query Difficult?, in Proceedings of SIGIR 2006. Google ScholarDigital Library
- C.L.A. Clarke, F. Scholer, I. Soboroff, The TREC 2005 Terabyte Track, In the Online Proceedings of 2005 TREC.Google Scholar
- B. He and I. Ounis. Inferring query performance using pre-retrieval predictors. In proceedings of the SPIRE 2004.Google ScholarCross Ref
- S. Tomlinson. Robust, Web and Terabyte Retrieval with Hummingbird SearchServer at TREC 2004. In the Online Proceedings of 2004 TREC.Google Scholar
- S. Cronen-Townsend, Y. Zhou, W.B. Croft, Predicting Query Performance, in Proceedings of SIGIR 2002. Google ScholarDigital Library
- V. Vinay, I. J.Cox, N. Mill-Frayling, K. Wood, On Ranking the Effectiveness of Searcher, in Proceedings of SIGIR 2006. Google ScholarDigital Library
- D. Metzler, W.B. Croft, A Markov Random Filed Model for Term Dependencies, in Proceedings of SIGIR 2005. Google ScholarDigital Library
- D. Metzler, T. Strohman, Y. Zhou, W.B. Croft, Indri at TREC 2005: Terabyte Track, In the Online Proceedings of 2004 TREC.Google Scholar
- P. Ogilvie and J. Callan, Combining document representations for known-item search, in Proceedings of SIGIR 2003. Google ScholarDigital Library
- A. Berger, J. Lafferty, Information retrieval as statistical translation, in Proceedings of SIGIR 1999. Google ScholarDigital Library
- Indri search engine: http://www.lemurproject.org/indri/Google Scholar
- I.J. Taneja: On Generalized Information Measures and Their Applications, Advances in Electronics and Electron Physics, Academic Press (USA), 76, 1989, 327--413.Google Scholar
- S. Cronen-Townsend, Y. Zhou and Croft, W.B., "A Framework for Selective Query Expansion," in Proceedings of CIKM 2004. Google ScholarDigital Library
- F. Song, W.B. Croft, A general language model for information retrieval, in Proceedings of SIGIR 1999. Google ScholarDigital Library
- Personal email contact with Vishwa Vinay and our own experiments.Google Scholar
- E. Yom-Tov, S. Fine, D. Carmel, A. Darlow, Learning to Estimate Query Difficulty Including Applications to Missing Content Detection and Distributed Information retrieval, in Proceedings of SIGIR 2005. Google ScholarDigital Library
Index Terms
- Query performance prediction in web search environments
Recommendations
Information Needs, Queries, and Query Performance Prediction
SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information RetrievalThe query performance prediction (QPP) task is to estimate the effectiveness of a search performed in response to a query with no relevance judgments. Existing QPP methods do not account for the effectiveness of a query in representing the underlying ...
BERT-QPP: Contextualized Pre-trained transformers for Query Performance Prediction
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge ManagementQuery Performance Prediction (QPP) is focused on estimating the difficulty of satisfying a user query for a certain retrieval method. While most state of the art QPP methods are based on term frequency and corpus statistics, more recent work in this ...
Predicting query performance
SIGIR '02: Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrievalWe develop a method for predicting query performance by computing the relative entropy between a query language model and the corresponding collection language model. The resulting clarity score measures the coherence of the language usage in documents ...
Comments