Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Discover Computing
Erschienen in: Discover Computing 6/2006

01.12.2006

Evaluating the effectiveness of content-oriented XML retrieval methods

verfasst von: Norbert Gövert, Norbert Fuhr, Mounia Lalmas, Gabriella Kazai

Erschienen in: Discover Computing | Ausgabe 6/2006

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Content-oriented XML retrieval approaches aim at a more focused retrieval strategy: Instead of retrieving whole documents, document components that are exhaustive to the information need while at the same time being as specific as possible should be retrieved. In this article, we show that the evaluation methods developed for standard retrieval must be modified in order to deal with the structure of XML documents. More precisely, the size and overlap of document components must be taken into account. For this purpose, we propose a new effectiveness metric based on the definition of a concept space defined upon the notions of exhaustiveness and specificity of a search result. We compare the results of this new metric by the results obtained with the official metric used in INEX, the evaluation initiative for content-oriented XML retrieval.
Vorheriger Artikel Swedish full text retrieval: Effectiveness of different combinations of indexing strategies with query terms
Nächster Artikel Precision prediction based on ranked list coherence

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
5
For example, the original TREC collection contains both newspaper articles (of the size of one or more kB) and a number of Federal Register documents (up to a few MB large) (Harman, 1993); treating both kinds of documents equally in evaluation is not appropriate from our point of view.
 
6
We make no explicit assumptions about users here, due to the fact that little is known about user behaviour when searching XML documents. However, the ongoing INEX interactive track is addressing this issue (Tombros et al., 2005).
 
7
In this article, the terms elements and components are used interchangeably.
 
8
Readers familiar with the classical IR literature will note that the terms ‘exhaustiveness’ and ‘specificity’ originally were introduced in the context of document indexing, where they referred to properties of the set of indexing terms assigned to a document (Lancaster, 1968); in contrast, we are regarding properties of a document (component) with respect to a query here.
 
9
In this paper, we use the term ‘topical exhaustiveness’ instead of ‘topical relevance’, in order to emphasize the two dimensions of relevance regarded here.
 
10
In INEX 2002, another but comparable definition of relevance was used, also based on two dimensions. The first dimension, topical relevance, corresponds to the exhaustiveness dimension defined in INEX 2003. The second dimension, coverage, is related to specificity. It has four values: no coverage, too small, too big and exact.
 
11
This comes from the fact that exhaustiveness remains or increases when going from a child element to its parent element, whereas specificity usually decreases in such a case—see Section 5.
 
12
In INEX 2003, the HyREX system developed in Duisburg-Essen was made available to participants for the topic creation phase, see http://​www.​is.​informatik.​uni-duisburg.​de/​projects/​hyrex/​.
 
13
We chose these two test sets since they differ in the nature of the assessments and the size of the runs; the INEX 2004 setting was similar to that of 2003.
 
Literatur
Baeza-Yates, R., Fuhr, N., & Maarek, Y. S. (Eds.) (2002). Proceedings of the SIGIR 2002 Workshop on XML and Information Retrieval.
Beaulieu, M., & Robertson, S. (1996). Evaluating interactive systems in TREC. Journal of the American Society for Information Science, 47(1), 85–94.
Chiaramella, Y., Mulhem, P., & Fourel, F. (1996). A model for multimedia information retrieval. Technical report, FERMI ESPRIT BRA 8134, University of Glasgow.
Cleverdon, C. W., Mills, J., & Keen, E. M. (1966). Factors determining the performance of indexing systems, vol. 2: Test results. Technical report, Aslib Cranfield Research Project, Cranfield, England.
Cooper, W. S. (1968). Expected search length: A single measure of retrieval effectiveness based on weak ordering action of retrieval systems. Journal of the American Society for Information Science, 19, 30–41.
Cosijn, E., & Ingwersen, P. (2000). Dimensions of relevance. Information Processing and Management, 36(4), 533–550.
Croft, W. B., Moffat, A., van Rijsbergen, C. J., Wilkinson, R., & Zobel, J. (Eds.) (1998). In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM.
Fuhr, N., & Lalmas, M. (2004). Report on the INEX 2003 Workshop. SIGIR Forum, 38(1).
Fuhr, N., Malik, S., & Lalmas, M. (2004b). Overview of the INitiative for the Evaluation of XML Retrieval (INEX) 2003. In Fuhr et al. (2004a), (pp. 1–11).
Harman, D. (1993). Overview of the First Text REtrieval Conference. In D. Harman (Ed.) The First Text REtrieval Conference (TREC-1), Gaithersburg, Md. 20899, National Institute of Standards and Technology Special Publication 500-207.
Jones, K. S., & van Rijsbergen, C. J. (1976). Information retrieval test collections. Journal of Documentation, 32(1), 59–75.
Kando, N., & Adachi, J. (2004). Report from the NTCIR workshop 3. SIGIR Forum, 38(1), 10–16.
Kazai, G. (2004). Report of the INEX 2003 Metrics working group. In Fuhr et al. (2004a) pp. 184–190.
Kazai, G., Lalmas, M., & de Vries, A. P. (2004). The overlap problem in content-oriented XML retrieval evaluation. In K. Jäarvelin, J. Allen, P. Bruza, & M. Sanderson (Eds.), Proceedings of the 27st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 72–79) New York, ACM.
Kekäläinen, J., & Järvelin, K. (2002). Using graded relevance sssessments in IR evaluation. Journal of the American Society for Information Science and Technology, 53(13).
Lancaster, F. W. (1968). Evaluation of the MEDLARS demand service. Report, National Library of Medicine, Bethesda, Maryland.
Peters, C., Braschler, M., Gonzalo, J., & Kluck, M. (Eds.) (2002). Evaluation of cross-language information retrieval systems (CLEF 2001). Vol. 2406 of Lecture Notes in Computer Science. Heidelberg et al., Springer.
Piwowarski, B., & Lalmas, M. (2004). Ensuring consistent and exhaustive relevance assessments for XML retrieval evaluation. In L. Gravano (Ed.), Proceedings of the 13th International Conference on Information and Knowledge Management. New York, ACM.
Raghavan, V. V., Bollmann, P., & Jung, G. S. (1989). A critical investigation of recall and precision as measures of retrieval system performance. ACM Transactions on Information Systems, 7(3), 205–229.
Salton, G. (Ed.) (1971). The SMART retrieval system—Experiments in automatic document processing. Englewood, Cliffs, New Jersey: Prentice Hall.
Salton, G., & McGill, M. J. (1983). Introduction to modern information retrieval. New York: McGraw-Hill.
Saracevic, T. (1995). Evaluation of evaluation in information retrieval. In: E. A. Fox, P. Ingwersen, and R. Fidel (Eds.), Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. (pp. 138–146), New York, ACM. ISBN 0-89791-714-6.
Saracevic, T. (1996). Relevance reconsidered. In P. Ingwersen and N. O. Pors (Eds.), In Proceedings of the 2nd International Conference on Conceptions of Library and Information Science (CoLIS 2), Oct. 13–16, 1996, (pp. 201–218).
Tombros, A., Larsen, B., & Malik, S. (2005). The Interactive Track at INEX 2004. In N. Fuhr, M. Lalmas, S. Malik, & Z. Szlavik (Eds.), Advances in XML Information Retrieval: Third International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2004, Dagstuhl Castle, Germany, Dec. 6–8, 2004, Revised Selected Papers, Vol. 3493. Springer-Verlag GmbH. http://​www.​springeronline.​com/​3-540-26166-4.
trec_eval (2002). Evaluation techniques and measures. In Voorhees and Harman, (2002), NIST.
Voorhees, E. M. (1998). Variations in relevance judgements and the measurement of retrieval effectiveness. In Croft et al. (1998), (pp. 315–323), ACM.
Voorhees, E. M., & Harman, D. K. (Eds.) (2002). The Tenth Text Retrieval Conference (TREC 2001). Gaithersburg, MD, USA: NIST.
Wong, S. K. M., & Yao, Y. Y. (1995). On modeling information retrieval with probabilistic inference. ACM Transactions on Information Systems, 13(1), 38–68.
Zobel, J. (1998). How reliable are the results of large-scale information retrieval experiments?. In Croft et al. (1998), (pp. 307–314), ACM.
Metadaten
Titel
Evaluating the effectiveness of content-oriented XML retrieval methods
verfasst von
Norbert Gövert
Norbert Fuhr
Mounia Lalmas
Gabriella Kazai
Publikationsdatum
01.12.2006
Verlag
Springer Netherlands
Erschienen in
Discover Computing / Ausgabe 6/2006
Print ISSN: 2948-2984
Elektronische ISSN: 2948-2992
DOI
https://doi.org/10.1007/s10791-006-9008-2

Weitere Artikel der Ausgabe 6/2006

Discover Computing 6/2006 Zur Ausgabe

OriginalPaper

Precision prediction based on ranked list coherence

OriginalPaper

Δ-distance: A family of dissimilarity metrics between images represented by multi-level feature vectors

OriginalPaper

Swedish full text retrieval: Effectiveness of different combinations of indexing strategies with query terms

OriginalPaper

A relatedness analysis of government regulations using domain knowledge and structural organization

Premium Partner

    Bildnachweise