nach oben

Erschienen in:

2013 | OriginalPaper | Buchkapitel

Human Computation for Information Retrieval

verfasst von : Christopher G. Harris, Padmini Srinivasan

Erschienen in: Handbook of Human Computation

Verlag: Springer New York

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Human computation techniques, such as crowdsourcing and games, have demonstrated their ability to accomplish portions of information retrieval (IR) tasks that machine-based techniques find challenging. Query refinement is one such IR task that may benefit from human involvement. We conduct an experiment that evaluates the contributions of participants from Amazon Mechanical Turk (N = 40). Each of our crowd participants is randomly assigned to use one of two query interfaces: a traditional web-based interface or a game-based interface. We ask each participant to manually construct queries to respond to a set of OHSUMED information needs and we calculate their resulting recall and precision. Those using a web interface are provided feedback on their initial queries and asked to use this information to reformulate their original queries. Game interface users are provided with instant scoring and asked to refine their queries based on their scores. In our experiment, crowdsourcing-based methods in general provide a significant improvement over machine algorithmic methods, and among crowdsourcing methods, games provide a better mean average precision (MAP) for query reformulations as compared to a non-game interface.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Human Computation in Electronic Literature

Nächstes Kapitel Human Computation-Enabled Network Analysis for a Systemic Credit Risk Rating

Pseudo relevance feedback, also known as blind relevance feedback, automates the manual part of relevance feedback through local document analysis. The pseudo relevance feedback method is to perform normal retrieval to find an initial set of most relevant documents, assume that the top “k” ranked documents are relevant, and then perform relevance feedback techniques as before under this assumption. Evidence suggests that this method tends to work better than global document analysis (Xu and Croft 1996).

Ageev M, Guo Q, Lagun D, Agichtein E (2011) Find it if you can: a game for modeling different types of web search success using interaction data. In: Proceedings of SIGIR’11. ACM, New York, pp 345–354

Allan J, Papka R, Lavrenko V (1998) On-line new event detection and tracking. In: Proceedings of SIGIR’98. ACM, New York, pp 37–45

Alonso O, Lease M (2011) Crowdsourcing for information retrieval: principles, methods, and applications. In: Proceedings of SIGIR’11. ACM, New York, pp 1299–1300

Alonso O, Mizzaro S (2012) Using crowdsourcing for TREC relevance assessment. Inf Process Manage 48(6):1053–1066CrossRef

Anick P (2003) Using terminological feedback for web search refinement: a log-based study. In: Proceedings of SIGIR’03. ACM, New York, pp 88–95

Buckley C, Voorhees EM (2004) Retrieval evaluation with incomplete information. In: Proceedings of SIGIR’04. ACM, New York, pp 25–32

Dasdan A, Drome C, Kolay S, Alpern M, Han A, Chi T, Hoover J, Davtchev I, Verma S (2009) Thumbs-Up: a game for playing to rank search results. In: Proceedings of the ACM SIGKDD workshop on human computation, Paris. ACM, New York, pp 36–37

Efthimiadis EN (2000) Interactive query expansion: a user-based evaluation in a relevance feedback environment. J Am Soc Inf Sci 51(11):989–1003CrossRef

Harris CG (2012) An evaluation of search strategies for user-generated video content. In: Proceedings of the WWW Workshop on Crowdsourcing Web Search (Lyon, France), pp 48–53

Harris CG, Srinivasan P (2012) Applying human computation mechanisms to information retrieval. Proc Am Soc Inf Sci Technol 49(1):1–10CrossRef

Harris CG, Srinivasan P (2013) Comparing crowd-based, game-based, and machine-based approaches in initial query and query refinement tasks. In: Advances in information retrieval. Springer, Berlin Heidelberg, pp 495–506

Hersh W, Buckley C, Leone T, Hickam D (1994) OHSUMED: an interactive retrieval evaluation and new large test collection for research. In: Proceedings of SIGIR’94. Springer, London, pp 192–201

Jones KS (1972) A statistical interpretation of term specificity and its application in retrieval. J Doc 28(1):11–21CrossRef

Law E, Ahn L von, Mitchell T (2009) Search war: a game for improving web search. In: Proceedings of the ACM SIGKDD workshop on human computation, Paris. ACM, New York, pp 31–31

Lease M, Yilmaz E (2012) Crowdsourcing for information retrieval. ACM, New York, SIGIR Forum 45:2 (January 2012), pp 66–75

McKibbon KA, Haynes RB, Walker Dilks CJ, Ramsden MF, Ryan NC, Baker L, Flemming T, Fitzgerald D (1990) How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches. Comput Biomed Res 23(6):583–593CrossRef

Milne D, Nichols DM, Witten IH (2008) A competitive environment for exploratory query expansion. In: Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries (JCDL’08). ACM, New York, pp 197–200

Robertson SE, Walker S, Jones S, Hancock-Beaulieu MM, Gatford M (1995) Okapi at TREC-3. NIST Special Publication SP-1995. Gaithersburg, Maryland, USA, pp 109–121

Ruthven I (2003) Re-examining the potential effectiveness of interactive query expansion. In: Proceedings of SIGIR’03. ACM, New York, pp 213–220

Spink A, Jansen BJ, Wolfram D, Saracevic T (2002) From e-sex to e-commerce: web search changes. Computer 35(3):107–109CrossRef

Strohman T, Metzler D, Turtle H, Croft WB (2005) Indri: a language model-based search engine for complex queries. In: Proceedings of the international conference on intelligence analysis, McLean, VA. Poster, 2–6 May 2005

Xu J, Croft WB (1996) Query expansion using local and global document analysis. In: Proceedings of SIGIR’96. ACM, New York, pp 4–11

Yan T, Kumar V, Ganesan D (2010) CrowdSearch: exploiting crowds for accurate real-time image search on mobile phones. In: Proceedings of MobiSys’10. ACM, New York, pp 77–90

Titel: Human Computation for Information Retrieval
verfasst von: Christopher G. Harris
Padmini Srinivasan
Verlag: Springer New York
Buch: Handbook of Human Computation
Print ISBN: 978-1-4614-8805-7

Electronic ISBN: 978-1-4614-8806-4

Copyright-Jahr: 2013
DOI: https://doi.org/10.1007/978-1-4614-8806-4_18

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner