Mounia Lalmas
Skip Abstract Section
Abstract
The INitiative for the Evaluation of XML retrieval (INEX) was set up in 2002 to establish an infrastructure and provide means, in the form of large test collections and appropriate scoring methods, for evaluating the effectiveness of content-oriented XML retrieval systems. This report provides an overview of the evaluation methodology developed in INEX from 2002 to 2006.
- C. L. Clarke. Range results in XML retrieval. In INEX 2005 Workshop on Element Retrieval Methodology, pages 4--5, 2005.Google Scholar
- L. Denoyer and P. Gallinari. The Wikipedia XML Corpus. SIGIR Forum, 40(1), 2006. Google ScholarDigital Library
- N. Fuhr, N. Gövert, G. Kazai, and M. Lalmas, editors. INitiative for the Evaluation of XML Retrieval (INEX). Proceedings of the First INEX Workshop (INEX 2002), ERCIM Workshop Proceedings, 2003.Google Scholar
- N. Fuhr, M. Lalmas, and S. Malik, editors. INitiative for the Evaluation of XML Retrieval (INEX). Proceedings of the Second INEX Workshop (INEX 2003), 2004.Google Scholar
- N. Fuhr, M. Lalmas, S. Malik, and G. Kazai, editors. Advances in XML Information Retrieval and Evaluation: Fourth Workshop of the INitiative for the Evaluation of XML Retrieval (INEX 2005). Springer, 2006. Google ScholarDigital Library
- N. Fuhr, M. Lalmas, S. Malik, and Z. Szlávik, editors. Advances in XML Information Retrieval, Third International Workshop of the Initiative for the Evaluation of XML Retrieval (INEX 2004). Springer, 2005. Google ScholarDigital Library
- N. Fuhr, M. Lalmas, and A. Trotman, editors. Comparative Evaluation of XML Information Retrieval Systems, 5th International Workshop of the Initiative for the Evaluation of XML Retrieval (INEX 2006), Springer, 2007. Google ScholarDigital Library
- N. Goevert, N. Fuhr, M. Lalmas, and G. Kazai. Evaluating the effectiveness of content-oriented XML retrieval methods. Journal of Information Retrieval, 9(6):699--722, 2006. Google ScholarDigital Library
- D. Hawking, E. M. Voorhees, N. Craswell, and P. Bailey. Overview of the TREC-8 Web Track. In TREC, 1999.Google Scholar
- K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM TOIS, 20(4):422--446, 2002. Google ScholarDigital Library
- J. Kamps, M. Marx, M. de Rijke, and B. Sigurbjörnsson. Structured queries in XML retrieval. In CIKM, pages 2--11, 2005. Google ScholarDigital Library
- G. Kazai. Choosing an Ideal Recall-Base for the Evaluation of the Focused Task: Sensitivity Analysis of the XCG Evaluation Measures. In Fuhr et al. {7}.Google Scholar
- G. Kazai and M. Lalmas. Notes on What to Measure in INEX. In INEX 2005 Workshop on Element Retrieval Methodology, Glasgow, July 2005.Google Scholar
- G. Kazai and M. Lalmas. eXtended Cumulated Gain Measures for the evaluation of content-oriented XML retrieval. ACM TOIS, 24(4), pages 503--542, 2006. Google ScholarDigital Library
- G. Kazai and M. Lalmas. INEX 2005 Evaluation Metrics. In Fuhr et al. {5}.Google Scholar
- G. Kazai, M. Lalmas, and A. P. de Vries. The overlap problem in content-orientedXML retrieval evaluation. In SIGIR, pages 72--79, 2004. Google ScholarDigital Library
- G. Kazai, S. Masood, and M. Lalmas. A study of the assessment of relevance for the INEX'02 test collection. In ECIR, pages 296--310, 2004.Google ScholarCross Ref
- J. Kekäläinen and K. Järvelin. Using graded relevance assessments in IR evaluation. JASIST, 53(13), 2002. Google ScholarDigital Library
- M. Lalmas, G. Kazai, J. Kamps, J. Pehcevski, B. Piwowarski, and S. Robertson. INEX 2006 Evaluation Measures. In Fuhr et al. {7}.Google Scholar
- M. Lalmas and A. Tombros. INEX 2002 - 2006: Understanding XML Retrieval Evaluation. In DELOS Conference on Digital Libraries, Tirrenia, Pisa, Italy, 2007. Google ScholarDigital Library
- P. Ogilvie and M. Lalmas. Investigating the exhaustivity dimension in content-oriented XML element retrieval evaluation. In CIKM, pages 84--93, 2006. Google ScholarDigital Library
- J. Pehcevski and J. A. Thom. Hixeval: Highlighting xml retrieval evaluation. In Advances in XML Information Retrieval and Evaluation: Fourth Workshop of the INitiative for the Evaluation of XML Retrieval (INEX 2005), 2006. Google ScholarDigital Library
- B. Piwowarski and G. Dupret. Evaluation in (XML) information retrieval: expected precision-recall with user modelling (eprum). In SIGIR, pages 260--267, 2006. Google ScholarDigital Library
- B. Piwowarski and M. Lalmas. Providing consistent and exhaustive relevance assessments for XML retrieval evaluation. In CIKM, pages 361--370. Google ScholarDigital Library
- V. V. Raghavan, P. Bollmann, and G. S. Jung. A critical investigation of recall and precision as measures of retrieval system performance. ACM TOIS, 7(3):205--229, 1989. Google ScholarDigital Library
- A. Tombros, S. Malik, and B. Larsen. Report on the INEX 2004 interactive track. ACM SIGIR Forum, 39(1):43--49, June 2005. Google ScholarDigital Library
- A. Trotman and S. Geva. Passage Retrieval and Other XML-Retrieval Tasks. In SIGIR 2006 Workshop on XML Element Retrieval Methodology, pages 43--50, 2006. Google ScholarDigital Library
- A. Trotman and M. Lalmas. Strict and vague interpretation of XML-retrieval queries. In SIGIR, pages 709--710, 2006. Google ScholarDigital Library
- A. Trotman and M. Lalmas. Why structural hints in queries do not help XML retrieval. In SIGIR, pages 711--712, 2006. Google ScholarDigital Library
- A. Trotman and B. Sigurbjornsson. Narrowed extended XPATHI (NEXI). In INEX 2004 Workshop Proceedings, pages 16--40, 2004. Google ScholarDigital Library
Index Terms
- Evaluating XML retrieval effectiveness at INEX
Recommendations
XML retrieval: what to retrieve?
SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrievalThe fundamental difference between standard information retrieval and XML retrieval is the unit of retrieval. In traditional IR, the unit of retrieval is fixed: it is the complete document. In XML retrieval, every XML element in a document is a ...
Evaluating the effectiveness of content-oriented XML retrieval methods
AbstractContent-oriented XML retrieval approaches aim at a more focused retrieval strategy: Instead of retrieving whole documents, document components that are exhaustive to the information need while at the same time being as specific as possible should ...
Multimedia retrieval at INEX 2006
Structured document retrieval allows for the retrieval of document fragments, i.e. XML elements, containing relevant information. The main INEX Ad Hoc track focuses on text-based XML element retrieval. Although text is dominantly present in most XML ...
Comments