Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Improving BERT-based Query-by-Document Retrieval with Multi-task Optimization Read chapter Read first chapter

2022 | OriginalPaper | Chapter

ranx: A Blazing-Fast Python Library for Ranking Evaluation and Comparison

Author : Elias Bassani

Published in: Advances in Information Retrieval

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper presents ranx, a Python evaluation library for Information Retrieval built on top of Numba. ranx provides a user-friendly interface to the most common ranking evaluation metrics, such as MAP, MRR, and NDCG. Moreover, it offers a convenient way of managing the evaluation results, comparing different runs, performing statistical tests between them, and exporting LaTeX tables ready to be used in scientific publications, all in a few lines of code. The efficiency brought by Numba, a just-in-time compiler for Python code, makes the adoption ranx convenient even for industrial applications.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Multilingual Topic Labelling of News Topics Using Ontological Mapping
next chapter DuoSearch: A Novel Search Engine for Bulgarian Historical Documents
Footnotes
Literature
1.
Aycock, J.: A brief history of just-in-time. ACM Comput. Surv. 35(2), 97–113 (2003)CrossRef
2.
Breuer, T., Ferro, N., Maistro, M., Schaer, P.: repro_eval: a python interface to reproducibility measures of system-oriented IR experiments. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12657, pp. 481–486. Springer, Cham (2021). https://​doi.​org/​10.​1007/​978-3-030-72240-1_​51CrossRef
3.
Burges, C.J.C., et al.: Learning to rank using gradient descent. In: ICML, ACM International Conference Proceeding Series, vol. 119, pp. 89–96. ACM (2005)
4.
Gysel, C.V., de Rijke, M.: Pytrec_eval: an extremely fast python interface to trec_eval. In: SIGIR, pp. 873–876. ACM (2018)
5.
Harman, D.: Information Retrieval Evaluation. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers, San Rafael (2011)
6.
7.
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20(4), 422–446 (2002)CrossRef
8.
Lam, S.K., Pitrou, A., Seibert, S.: Numba: a llvm-based python JIT compiler. In: LLVM@SC, pp. 7:1–7:6. ACM (2015)
9.
Lucchese, C., Muntean, C.I., Nardini, F.M., Perego, R., Trani, S.: Rankeval: an evaluation and analysis framework for learning-to-rank solutions. In: SIGIR, pp. 1281–1284. ACM (2017)
10.
Lucchese, C., Muntean, C.I., Nardini, F.M., Perego, R., Trani, S.: Rankeval: evaluation and investigation of ranking models. SoftwareX 12, 100614 (2020)
11.
MacAvaney, S., Yates, A., Feldman, S., Downey, D., Cohan, A., Goharian, N.: Simplified data wrangling with ir_datasets. In: SIGIR, pp. 2429–2436. ACM (2021)
12.
Macdonald, C., Tonellotto, N.: Declarative experimentation in information retrieval using pyterrier. In: ICTIR, pp. 161–168. ACM (2020)
13.
Macdonald, C., Tonellotto, N., MacAvaney, S., Ounis, I.: Pyterrier: declarative experimentation in python from BM25 to dense retrieval. In: CIKM, pp. 4526–4533. ACM (2021)
14.
McKinney, W., et al.: Pandas: a foundational python library for data analysis and statistics. Python High Perf. Sci. Comput. 14(9), 1–9 (2011)
15.
Oliphant, T.E.: A guide to NumPy, vol. 1. Trelgol Publishing USA (2006)
16.
Oliphant, T.E.: Python for scientific computing. Comput. Sci. Eng. 9(3), 10–20 (2007)CrossRef
17.
Palotti, J.R.M., Scells, H., Zuccon, G.: Trectools: an open-source python library for information retrieval practitioners involved in trec-like campaigns. In: SIGIR, pp. 1325–1328. ACM (2019)
18.
Sanderson, M.: Test collection based evaluation of information retrieval systems. Found. Trends Inf. Retr. 4(4), 247–375 (2010)CrossRefMATH
19.
Smucker, M.D., Allan, J., Carterette, B.: A comparison of statistical significance tests for information retrieval evaluation. In: CIKM, pp. 623–632. ACM (2007)
20.
Van Rossum, G., Drake Jr, F.L.: Python reference manual. Centrum voor Wiskunde en Informatica Amsterdam (1995)
21.
Voorhees, E., Harman, D.: Experiment and evaluation in information retrieval (2005)
22.
van der Walt, S., Colbert, S.C., Varoquaux, G.: The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)
Metadata
Title
ranx: A Blazing-Fast Python Library for Ranking Evaluation and Comparison
Author
Elias Bassani
Copyright Year
2022
Book
Advances in Information Retrieval

Print ISBN: 978-3-030-99738-0

Electronic ISBN: 978-3-030-99739-7

Copyright Year: 2022

DOI
https://doi.org/10.1007/978-3-030-99739-7_30