Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Digital Libraries Kapitel lesen Erstes Kapitel lesen

2011 | OriginalPaper | Buchkapitel

4. The User in Interactive Information Retrieval Evaluation

verfasst von : Peter Ingwersen

Erschienen in: Advanced Topics in Information Retrieval

Verlag: Springer Berlin Heidelberg

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This chapter initially defines what characterizes and distinguishes research frameworks from research models. The Laboratory Research Framework for IR illustrates the case. We define briefly what is meant by the concept of research design, including research questions, and what this chapter regards as central IIR evaluation research settings and variables. This is followed by a description of IIR components, pointing to the elements of the Integrated Cognitive Research Framework for IR that incorporates the Laboratory Framework in a contextual manner. The following sections describe and exemplify: (1) Request types, test persons, task-based simulations of search situations and relevance or performance measures in IIR; (2) Ultra-Light Interactive IR experiments; (3) Interactive-Light IR studies; and (4) Naturalistic field investigations of IIR. The chapter concludes with a summary section, a reference list and a thematically classified bibliography.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel Spam, Opinions, and Other Relationships: Towards a Comprehensive View of the Web Knowledge Discovery
Nächstes Kapitel Aggregated Search
Literatur
Anick P (2003) Using terminological feedback for web search refinement: A log based study. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 88–95
Baeza-Yates R, Ribeiro-Neto B (1999) Modern Information Retrieval. ACM Press/Addison-Wesley, New York, NY
Barry CL (1994) User-defined relevance criteria: An exploratory study. Journal of the American Society for Information Science 45:149–159 CrossRef
Barry CL, Schamber L (1998) User’s criteria for relevance evaluation: A cross situational comparison. Information Processing and Management 34(2/3):219–236 CrossRef
Beaulieu M (1997) Experiments with interfaces to support query expansion. Journal of Documentation 53:8–19 CrossRef
Beaulieu M (2000) Interaction in information searching and retrieval. Journal of Documentation 431–439
Beaulieu M, Jones S (1998) Interactive searching and interface issues in the okapi best match probabilistic retrieval system. Interacting with Computers 10:237–248 CrossRef
Belew R (2000) Finding out About: A Cognitive Perspective on Search Engine Technology and the WWW. Cambridge University Press, Cambridge MATH
Belkin N, Vickery A (1985) Interaction in information systems. British Library, London
Belkin N, Oddy R, Brooks H (1982) Ask for information retrieval: Part 2. Journal of Documentation 38:145–164 CrossRef
Belkin N, Seeger T, Wersig G (1983) Distributed expert problem treatment as a model for information systems analysis and design. Journal of Information Science: Principles and Practice 5:153–167
Belkin N, Cool C, Croft W, Callan J (1993) The effect of multiple query representations on information retrieval performance. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 339–346
Belkin N, Cool C, Koenemann J, Park S, Ng W (1996a) Information seeking behaviour in new searching environments. In: Ingwersen P, Pors N (eds) Proceedings of the Conference on Conceptions of Library and Information Science, pp 403–416
Belkin N, Cool C, Koenemann J, Ng W, Park S (1996b) Using relevance feedback and ranking in interactive searching. In: Harman D (ed) Proceedings of the Text Retrieval Conference, pp 181–210
Belkin N, Cool C, Kelly D, Lee HJ, Muresan G, Tang MC, Yuan XJ (2003) Query length in interactive information retrieval. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 205–212
Bellotti V, Ducheneaut N, Howard M, Smith I (2003) Taking email to task: The design and evaluation of a task management centered email tool. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp 345–352 CrossRef
Bilal D (2000) Children’s use of the Yahooligans! web search engine: 1. Cognitive, physical, and affective behaviors on fact-based search tasks. Journal of the American Society for Information Science, pp 1170–1183
Borlund P (2000) Experimental components for the evaluation of interactive information retrieval systems. Journal of Documentation 56:71–90 CrossRef
Borlund P, Ingwersen P (1998) Measures of relative relevance and ranked half-life: performance indicators for interactive IR. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 324–331. http://​doi.​acm.​org/​10.​1145/​290941.​291019, visited on December, 2010
Buckland L, Voorhees E (eds) (2005) Proceedings of the Text Retrieval Conference. Government Printing Office, Washington, DC
Croft W, Thompson R (1987) I3r: a new approach to the design of document retrieval systems. Journal of the American Society for Information Science 38(6):389–404 CrossRef
Czerwinski M, Horvitz E, Cutrell E (2001) Subjective duration assessment: An implicit probe for software usability. In: Proceedings of the Conference on Interaction Homme-Machine and Human-Computer Interaction, pp 167–170
Dumais S, Cutrell E, Cadiz J, Jancke G, Sarin R, Robbins D (2003) Stuff i’ve seen: A system for personal information retrieval and re-use. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 72–79
Efron M (2009) Using multiple query aspects to build test collections without human relevance judgments. In: Proceedings of the European Conference on Information Retrieval. Springer, Heidelberg, pp 276–287
Efthimiadis E (1993) A user-centered evaluation of ranking algorithms for interactive query expansion. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 146–156
Ericsson K, Simon H (1996) Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge, MA
Fidel R (1993) Qualitative methods in information retrieval research. Library and Information Science Research 15:219–247
Fidel R, Soergel D (1983) Factors affecting online bibliographic retrieval: A conceptual framework for research. Journal of the American Society for Information Science 34:163–180 CrossRef
Fidel R, Davies RK, Douglass MH, Holder JK, Hopkins CJ, Kushner EJ, Miyagishima BK, Toney CD (1999) A visit to the information mall: Web searching behavior of high school students. Journal of the American Society for Information Science 50:24–37 CrossRef
Ford N, Miller D, Moss N (2001) The role of individual differences in internet searching: An empirical study. Journal of the American Society for Information Science and Technology 52:1049–1066 CrossRef
Frankfort-Nachmias C, Nachmias D (2000) Research Methods in the Social Sciences. Worth Publishers, New York
Frokjaer E, Hertzum M, Hormbaek K (2000) Measuring usability: Are effectiveness and efficiency and satisfaction really correlated? In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp 345–352 CrossRef
Harman D (ed) (1996) Proceedings of the Text Retrieval Conference. Government Printing Office, Washington, DC
Harper D, Koychev I, Sun Y, Pirie I (2004) Within-document retrieval: A user-centred evaluation of relevance profiling. Journal of Information Retrieval 7:265–290 CrossRef
Hawking D, Voorhees E, Craswell N, Bailey P (2000) Overview of the TREC web track. In: Voorhees E, Harman D (eds) Proceedings of the Text Retrieval Conference, pp 131–150
Hersh W, Pentecost J, Hickam D (1996) A task oriented approach to information retrieval evaluation. Journal of the American Society for Information Science 47:50–56 CrossRef
Hirsh S (1999) Children’s relevance criteria and information-seeking on electronic resources. Journal of the American Society for Information Science 50:1265–1283 CrossRef
Hornbaek K (2006) Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies 64:79–102 CrossRef
Hsieh-Yee I (1998) Search tactics of web users in searching for texts, graphics, known items and subjects: A search simulation study. Reference Librarian 60:61–85 CrossRef
Ingwersen P (1992) Information Retrieval Interaction. Taylor Graham, London
Ingwersen P, Järvelin K (2005) The Turn: Integration of Information Seeking and Retrieval in Context. Springer, Berlin MATH
Ingwersen P, Järvelin K (2007) On the holistic cognitive theory for information retrieval: Drifting outside the cave of the laboratory framework. In: Dominich S, Kisss F (eds) Studies in Theory of Information Retrieval. Foundation for Information Society, pp 135–147
Ingwersen P, Willett P (1995) An introduction to algorithmic and cognitive approaches for information retrieval. Libri 45:160–177 CrossRef
Järvelin K (2007) An analysis of two approaches in information retrieval: From frameworks to study designs. Journal of the American Society for Information Science and Technology 58:971–986 CrossRef
Järvelin K, Kekäläinen J (2000) IR evaluation methods for highly relevant documents. In: Belkin N, Ingwersen P, Leong MK (eds) Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 41–48
Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20:422–446 CrossRef
Joachims T, Granka L, Pan B, Hembrooke H, Gay G (2005a) Accurately interpreting click-through data as implicit feedback. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 154–161
Käki M (2004) Proportional search interface usability measures. In: Proceedings of the Nordic Forum for Human-Computer Interaction Research, pp 365–372
Kekäläinen J (2005) Binary and graded relevance in IR evaluations—comparison of the effects on ranking of IR systems. Information Processing and Management 41:1019–1033 CrossRef
Kekäläinen J, Järvelin K (2002b) Using graded relevance assessments in IR evaluation. Journal of the American Society for Information Science and Technology 53:1120–1129 CrossRef
Kekäläinen J, Järvelin K (2002a) User-oriented evaluation methods for information retrieval: A case study based on conceptual models for query expansion. In: Lakemeyer G, Nebel B (eds) Exploring Artificial Intelligence in the New Millennium. Morgan Kaufmann, San Diego, CA, pp 355–379
Kelly D, Belkin N (2004) Display time as implicit feedback: Understanding task effects. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 377–384
Kelly D, Fu X (2007) Eliciting better information need descriptions from users of information search systems. Information Processing and Management 43:30–46 CrossRef
Kelly D, Dollu V, Fu X (2005) The loquacious user: A document-independent source of terms for query expansion. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 457–464
Koenemann J, Belkin N (1996) A case for interaction: A study of interactive information retrieval behavior and effectiveness. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp 205–212
Larsen SM, Tombros A (2006) The interactive track at INEX. In: Fuhr N, Lalmas M, Malik S, Kazai G (eds) Proceedings of the Workshop of Initiative for the Evaluation of XML Retrieval. Springer, Berlin, pp 398–410 CrossRef
Lykke M (2001) A framework for work task based thesaurus design. Journal of Documentation 57:774–797 CrossRef
Lykke M (2004) Task-based evaluation of associative thesaurus in real-life environment. In: Proceedings of the Meeting of the American Society for Information Science and Technology, pp 437–447
Magennis M, van Rijsbergen K (1997) The potential and actual effectiveness of interactive query expansion. In: Belkin N, Narasimhalu A, Willett P (eds) Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 324–332
Maglaughlin K, Sonnenwald D (2002) User perspectives on relevance criteria: A comparison among relevant, partially relevant and not-relevant judgments. Journal of the American Society for Information Science and Technology 53:327–342 CrossRef
Palmquist R, Kim K (2000) Cognitive style and on line database search experience as predictors of web search performance. Journal of the American Society for Information Science 51:558–566 CrossRef
Papaeconomou C, Zijlema AF, Ingwersen P (2008) Searchers’ relevance judgments and criteria in evaluating web pages in a learning style perspective. In: Proceedings of the International Symposium on Information Interaction in Context. IIiX ’08. ACM Press, New York, NY, pp 123–132. http://​doi.​acm.​org/​10.​1145/​1414694.​1414722, visited on December, 2010 CrossRef
Petrelli D, Beaulieu M, Sanderson G, Demetriou M, Herring P, Hansen P (2004) Observing users and designing clarity: A case study on the user-centered design of a cross-language information retrieval system. Journal of the American Society for Information Science and Technology 55:923–934 CrossRef
Puolamäki K, Salojärvi J, Savia E (2005) Combining eye movements and collaborative filtering for proactive information retrieval. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 146–153
Ruthven I, Lalmas M, van Rijsbergen K (2002) Ranking expansion terms with partial and ostensive evidence. In: Proceedings of the Conference on Conceptions of Library and Information Science, pp 199–220
Ruthven I, Lalmas M, van Rijsbergen K (2003) Incorporating user search behaviour into relevance feedback. Journal of the American Society for Information Science and Technology 54:529–549 CrossRef
Salojärvi J, Kojo I, Simola J, Kaski S (2003) Can relevance be inferred from eye movements in information retrieval? In: Proceedings of the Workshop on Self-Organizing Maps, Kitakyushu, Japan, pp 261–266
Sanderson M, Zobel J (2005) Information retrieval system evaluation: Effort, sensitivity and reliability. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 162–169
Saracevic T, Kantor P (1988a) A study of information seeking and retrieving: II. Users, questions and effectiveness. Journal of the American Society for Information Science 39:177–196 CrossRef
Saracevic T, Kantor P (1988b) A study of information seeking and retrieving: III. Searchers, searches, and overlaps. Journal of the American Society for Information Science 39:197–216 CrossRef
Saracevic T, Kantor P, Chamis A, Trivison D (1988) A study of information seeking and retrieving: I. Background and methodology. Journal of the American Society for Information Science 39:161–176 CrossRef
Sormunen E (2002) Liberal relevance criteria of TREC—counting on negligible documents? In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 320–330
Spink A, Greisdorf H, Bateman J (1998) From highly relevant to not relevant: Examining different regions of relevance. Information Processing and Management 34:599–621 CrossRef
Tague J, Schultz R (1988) Some measures and procedures for evaluation of the userinterface in an information retrieval system. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 371–385
Teevan J, Alvarado C, Ackerman M, Karger D (2004) The perfect search engine is not enough: A study of orienteering behavior in directed search. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp 415–422 CrossRef
Tombros A, Ruthven I, Jose J (2003) Searchers’ criteria for assessing web pages. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 96–104
Vakkari P (2001) Changes in search tactics and relevance judgments in preparing a research proposal: A summary of findings of a longitudinal study. Journal of Information Retrieval 4(3/4):295–310 MATH
Vakkari P, Hakala N (2000) Changes in relevance criteria and problem stages in task performance. Journal of Documentation 56:540–562 CrossRef
Vakkari P, Sormunen E (2004) The influence of relevance levels on the effectiveness of interactive information retrieval. Journal of the American Society for Information Science and Technology 55:963–969 CrossRef
Voorhees E (1998) Variations in relevance judgments and the measurement of retrieval effectiveness. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 315–323
Wang P, Hawk W, Tenopir C (2000) Users’ interaction with World Wide Web resources: An exploratory study using a holistic approach. Information Processing and Management 36:229–251 CrossRef
White R (2006) Using searcher simulations to redesign a polyrepresentative implicit feedback interface. Information Processing and Management 42:1185–1202 CrossRef
White R, Jose JM, Ruthven I (2003) A task-oriented study of the influencing effects of query-biased summarization in web searching. Information Processing and Management 39:707–733 MATHCrossRef
White R, Ruthven I, Jose J (2005a) A study of factors affecting the utility of implicit relevance feedback. In: Proceedings of the ACM Conference on Research and Development in Information Retrieval. ACM Press, New York, NY, pp 35–42
White R, Ruthven I, Jose J, van Rijsbergen K (2005b) Evaluating implicit feedback models using searcher simulations. ACM Transactions on Information Systems 325–361
White R, Jose J, Ruthven I (2006) An implicit feedback approach for interactive information retrieval. Information Processing and Management 42:166–190 CrossRef
Metadaten
Titel
The User in Interactive Information Retrieval Evaluation
verfasst von
Peter Ingwersen
Copyright-Jahr
2011
Verlag
Springer Berlin Heidelberg
Buch
Advanced Topics in Information Retrieval

Print ISBN: 978-3-642-20945-1

Electronic ISBN: 978-3-642-20946-8

Copyright-Jahr: 2011

DOI
https://doi.org/10.1007/978-3-642-20946-8_4

Neuer Inhalt

    Bildnachweise