Zhe Cao
Ming-Feng Tsai
Hang Li
ABSTRACT
The paper is concerned with learning to rank, which is to construct a model or a function for ranking objects. Learning to rank is useful for document retrieval, collaborative filtering, and many other applications. Several methods for learning to rank have been proposed, which take object pairs as 'instances' in learning. We refer to them as the pairwise approach in this paper. Although the pairwise approach offers advantages, it ignores the fact that ranking is a prediction task on list of objects. The paper postulates that learning to rank should adopt the listwise approach in which lists of objects are used as 'instances' in learning. The paper proposes a new probabilistic method for the approach. Specifically it introduces two probability models, respectively referred to as permutation probability and top k probability, to define a listwise loss function for learning. Neural Network and Gradient Descent are then employed as model and algorithm in the learning method. Experimental results on information retrieval show that the proposed listwise approach performs better than the pairwise approach.
- Baeza-Yates, R., & Ribeiro-Neto, B. (1999). Modern information retrieval. Addison Wesley. Google Scholar
Digital Library
- Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005). Learning to rank using gradient descent. Proceedings of ICML 2005 (pp. 89--96). Google ScholarDigital Library
- Cao, Y. B., Xu, J., Liu, T. Y., Li, H., Huang, Y. L., & Hon, H. W. (2006). Adapting ranking SVM to document retrieval. Proceedings of SIGIR 2006 (pp. 186--193). Google ScholarDigital Library
- Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H. (2007). Learning to rank: From pairwise approach to listwise approach (Technical Report MSR-TR-2007-40). Microsoft Corporation.Google ScholarDigital Library
- Cohen, W. W., Schapire, R. E., & Singer, Y. (1998). Learning to order things. Advances in Neural Information Processing Systems. The MIT Press. Google ScholarDigital Library
- Crammer, K., & Singer, Y. (2001). Pranking with ranking. Proceedings of NIPS 2001.Google Scholar
- Craswell, N., Hawking, D., Wilkinson, R., & Wu, M. (2003). Overview of the TREC 2003 web track. Proceedings of TREC 2003 (pp. 78--92).Google Scholar
- Freund, Y., Iyer, R., Schapire, R. E., & Singer, Y. (1998). An efficient boosting algorithm for combining preferences. Proceedings of ICML 1998 (pp. 170--178). Google ScholarDigital Library
- Herbrich, R., Graepel, T., & Obermayer, K. (1999). Support vector learning for ordinal regression. Proceedings of ICANN 1999 (pp. 97--102).Google ScholarCross Ref
- Hersh, W. R., Buckley, C., Leone, T. J., & Hickam, D. H. (1994). OHSUMED: An interactive retrieval evaluation and new large test collection for research. Proceedings of SIGIR 1994 (pp. 192--201). Google ScholarDigital Library
- Jarvelin, K., & Kekanainen, J. (2000). IR evaluation methods for retrieving highly relevant documents. Proceedings of SIGIR 2000 (pp. 41--48). Google ScholarDigital Library
- Joachims, T. (1999). Making large-scale support vector machine learning practical. Advances in kernel methods: support vector learning, 169--184. Google ScholarDigital Library
- Joachims, T. (2002). Optimizing search engines using clickthrough data. Proceedings of KDD 2002 (pp. 133--142). Google ScholarDigital Library
- Lebanon, G., & Lafferty, J. (2002). Cranking: Combining rankings using conditional probability models on permutations. Proceedings of ICML 2002 (pp. 363--370). Google ScholarDigital Library
- Luce, R. D. (1959). Individual choice behavior. Wiley.Google Scholar
- Nallapati, R. (2004). Discriminative models for information retrieval. Proceedings of SIGIR 2004 (pp. 64--71). Google ScholarDigital Library
- Plackett, R. L. (1975). The analysis of permutations. Applied Statistics, 24(2), 193--202.Google ScholarCross Ref
- Qin, T., Liu, T.-Y., Lai, W., Zhang, X.-D., Wang, D.-S., & Li, H. (2007). Ranking with multiple hyperplanes. Proceedings of SIGIR 2007. Google ScholarDigital Library
- Shashua, A., & Levin, A. (2002). Taxonomy of large margin principle algorithms for ordinal regression problems. Proceedings of NIPS 2002.Google Scholar
- Tsai, M.-F., Liu, T.-Y., Qin, T., Chen, H.-H., & Ma, W.-Y. (2007). Frank: A ranking method with fidelity loss. Proceedings of SIGIR 2007. Google ScholarDigital Library
- Learning to rank: from pairwise approach to listwise approach
Recommendations
Learning to re-rank: query-dependent image re-ranking using click data
WWW '11: Proceedings of the 20th international conference on World wide webOur objective is to improve the performance of keyword based image search engines by re-ranking their original results. To this end, we address three limitations of existing search engines in this paper. First, there is no straight-forward, fully ...
Unbiased Learning to Rank: Theory and Practice
ICTIR '18: Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information RetrievalImplicit user feedback (such as clicks and dwell time) is an important source of data for modern search engines. While heavily biased~\citejoachims2005accurately,keane2006modeling,joachims2007evaluating,yue2010beyond, it is cheap to collect and ...
Learning to rank code examples for code search engines
Source code examples are used by developers to implement unfamiliar tasks by learning from existing solutions. To better support developers in finding existing solutions, code search engines are designed to locate and rank code examples relevant to user'...
Comments