Nicola Ferro
Claudio Lucchese
Raffaele Perego
ABSTRACT
Ranking query results effectively by considering user past behaviour and preferences is a primary concern for IR researchers both in academia and industry. In this context, LtR is widely believed to be the most effective solution to design ranking models that account for user-interaction features that have proved to remarkably impact on IR effectiveness. In this paper, we explore the possibility of integrating the user dynamic directly into the LtR algorithms. Specifically, we model with Markov chains the behaviour of users in scanning a ranked result list and we modify Lambdamart, a state-of-the-art LtR algorithm, to exploit a new discount loss function calibrated on the proposed Markovian model of user dynamic. We evaluate the performance of the proposed approach on publicly available LtR datasets, finding that the improvements measured over the standard algorithm are statistically significant.
- E. Agichtein, E. Brill, and S. Dumais. Improving Web Search Ranking by Incorporating User Behavior Information. In SIGIR, pages 19--26, ACM, 2006. Google Scholar
Digital Library
- A. Broder. A Taxonomy of Web Search. SIGIR Forum, 36(2):3--10, 2002. Google ScholarDigital Library
- C. J. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to Rank Using Gradient Descent. In ICML, pages 89--96, ACM, 2005. Google ScholarDigital Library
- C. J. C. Burges. From RankNet to LambdaRank to LambdaMART: An Overview. Technical Report, 2010.Google Scholar
- C. J. C. Burges, R. Ragno, and Q. V. Le. Learning to Rank with Nonsmooth Cost Functions. In NIPS, Vol. 6, pages 193--200, 2006.Google Scholar
- G. Capannini, C. Lucchese, F. M. Nardini, S. Orlando, R. Perego and N. Tonellotto. Quality versus Efficiency in Document Scoring with Learning-to-Rank Models. In IPM, 52(6):1161--1177, 2016. Google ScholarDigital Library
- O. Chapelle, T. Joachims, F. Radlinski, and Y. Yue. Large-scale Validation and Analysis of Interleaved Search Evaluation. In TOIS, 30(1):6:1--6:41, 2012. Google ScholarDigital Library
- D. Dato, C. Lucchese, F. M. Nardini, S. Orlando, R. Perego, N. Tonellotto, and R. Venturini. Fast Ranking with Additive Ensembles of Oblivious and Non-Oblivious Regression Trees. In TOIS, 35(2):15:1--15:31, 2016. Google ScholarDigital Library
- P. Donmez, K. M. Svore, and C. J. C. Burges. On the Local Optimality of LambdaRank. In SIGIR, pages 460--467, ACM, 2009. Google ScholarDigital Library
- M. Ferrante, N. Ferro, and M. Maistro. Injecting User Models and Time into Precision via Markov Chains. In SIGIR, pages 597--606, ACM, 2014. Google ScholarDigital Library
- K. Hofmann, A. Schuth, S. Whiteson, and M. de Rijke. Reusing Historical Interaction Data for Faster Online Learning to Rank for IR. In WSDM, pages 183--192, ACM, 2013. Google ScholarDigital Library
- T. Liu. Learning to Rank for Information Retrieval. IN FnTIR, 3(3):225--331, 2009.Google Scholar
- C. Lucchese, S. Orlando, R. Perego, F. Silvestri and G. Tolomei. Discovering tasks from search engine query logs. In TOIS, 31(3):14:1--14:43, 2013. Google ScholarDigital Library
- J. R. Norris. Markov Chains. Cambridge University Press, 1998.Google Scholar
- T. Qin, and T. Liu. Introducing LETOR 4.0 Datasets. In CoRR, 2013.Google Scholar
- A. Schuth, K. Hofmann, S. Whiteson, and M. de Rijke. Lerot: An Online Learning to Rank Framework. In LivingLab, pages 23--26, ACM, 2013.Google ScholarDigital Library
- P. Serdyukov, N. Craswell, G. Dupret. WSCD2012: Workshop on Web Search Click Data 2012. In WSDM, pages 771--772, ACM, 2012.Google Scholar
- I. Teodorescu. Maximum Likelihood Estimation for Markov Chains. In arXiv preprint arXiv:0905.4131, 2009.Google Scholar
- Q. Wu, Burges, C. J. C. Christopher K. M. Svore, J. Gao. Adapting boosting for information retrieval measures. In Information Retrieval, 13(3):254--270, 2010. Google ScholarDigital Library
- E. Yilmaz, M. Shokouhi, N. Craswell, and S. Robertson. Expected Browsing Utility for Web Search Evaluation. In CIKM, pages 1561--1565, ACM, 2010. Google ScholarDigital Library
- Y. Zhang, L. A. F. Park, and A. Moffat. Click-based Evidence for Decaying Weight Distributions in Search Effectiveness Metrics. In Information Retrieval, 13(1):46--69, 2010. Google ScholarDigital Library
Index Terms
- On Including the User Dynamic in Learning to Rank
Recommendations
Unbiased LambdaMART: An Unbiased Pairwise Learning-to-Rank Algorithm
WWW '19: The World Wide Web ConferenceRecently a number of algorithms under the theme of 'unbiased learning-to-rank' have been proposed, which can reduce position bias, the major type of bias in click data, and train a high-performance ranker with click data. Most of the existing algorithms,...
Learning to rank code examples for code search engines
Source code examples are used by developers to implement unfamiliar tasks by learning from existing solutions. To better support developers in finding existing solutions, code search engines are designed to locate and rank code examples relevant to user'...
On Application of Learning to Rank for E-Commerce Search
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information RetrievalE-Commerce (E-Com) search is an emerging important new application of information retrieval. Learning to Rank (LETOR) is a general effective strategy for optimizing search engines, and is thus also a key technology for E-Com search. While the use of ...
Comments