Skip to main content
Top
Published in:
Cover of the book

2019 | OriginalPaper | Chapter

What Happened in CLEF\(\ldots \) For a While?

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

2019 marks the 20\(^\text {th}\) birthday for CLEF, an evaluation campaign activity which has applied the Cranfield evaluation paradigm to the testing of multilingual and multimodal information access systems in Europe. This paper provides a summary of the motivations which led to the establishment of CLEF, and a description of how it has evolved over the years, the major achievements, and what we see as the next challenges.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Abnar, S., Dehghani, M., Shakery, A.: Meta text aligner: text alignment based on predicted plagiarism relation. In: Mothe et al. [301], pp. 193–199 Abnar, S., Dehghani, M., Shakery, A.: Meta text aligner: text alignment based on predicted plagiarism relation. In: Mothe et al. [301], pp. 193–199
2.
go back to reference Adams, J., Bedrick, S.: Automatic indexing of journal abstracts with latent semantic analysis. In: Mothe et al. [301], pp. 200–208 Adams, J., Bedrick, S.: Automatic indexing of journal abstracts with latent semantic analysis. In: Mothe et al. [301], pp. 200–208
3.
go back to reference Afzal, Z., Akhondi, S.A., van Haagen, H.H.H.B.M., van Mulligen, E.M., Kors, J.A.: Concept recognition in French biomedical text using automatic translation. In: Fuhr et al. [143], pp. 162–173 Afzal, Z., Akhondi, S.A., van Haagen, H.H.H.B.M., van Mulligen, E.M., Kors, J.A.: Concept recognition in French biomedical text using automatic translation. In: Fuhr et al. [143], pp. 162–173
4.
go back to reference Agirre, E., Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2008: ad hoc track overview. In: Peters et al. [359], pp. 15–37 Agirre, E., Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2008: ad hoc track overview. In: Peters et al. [359], pp. 15–37
5.
go back to reference Agirre, E., Di Nunzio, G.M., Mandl, T., Otegi, A.: CLEF 2009 ad hoc track overview: robust-WSD task. In: Peters et al. [360], pp. 36–49 Agirre, E., Di Nunzio, G.M., Mandl, T., Otegi, A.: CLEF 2009 ad hoc track overview: robust-WSD task. In: Peters et al. [360], pp. 36–49
6.
go back to reference Agirre, E., de Lacalle, O.L., Magnini, B., Otegi, A., Rigau, G., Vossen, P.: SemEval-2007 task 01: evaluating WSD on cross-language information retrieval. In: Peters et al. [366], pp. 908–917 Agirre, E., de Lacalle, O.L., Magnini, B., Otegi, A., Rigau, G., Vossen, P.: SemEval-2007 task 01: evaluating WSD on cross-language information retrieval. In: Peters et al. [366], pp. 908–917
7.
go back to reference Agosti, M., Di Buccio, E., Ferro, N., Masiero, I., Peruzzo, S., Silvello, G.: DIRECTions: design and specification of an IR evaluation infrastructure. In: Catarci et al. [75], pp. 88–99 Agosti, M., Di Buccio, E., Ferro, N., Masiero, I., Peruzzo, S., Silvello, G.: DIRECTions: design and specification of an IR evaluation infrastructure. In: Catarci et al. [75], pp. 88–99
8.
go back to reference Agosti, M., Di Nunzio, G.M., Ferro, N., Silvello, G.: An innovative approach to data management and curation of experimental data generated through IR test collections. In: Ferro and Peters [130] Agosti, M., Di Nunzio, G.M., Ferro, N., Silvello, G.: An innovative approach to data management and curation of experimental data generated through IR test collections. In: Ferro and Peters [130]
10.
go back to reference Carrillo-de Albornoz, J., Gonzalo, J., Amigó, E.: RepLab: an evaluation campaign for online monitoring systems. In: Ferro and Peters [130] Carrillo-de Albornoz, J., Gonzalo, J., Amigó, E.: RepLab: an evaluation campaign for online monitoring systems. In: Ferro and Peters [130]
11.
go back to reference Alfalahi, A., Eriksson, G., Sneiders, E.: Shadow answers as an intermediary in email answer retrieval. In: Mothe et al. [301], pp. 209–214 Alfalahi, A., Eriksson, G., Sneiders, E.: Shadow answers as an intermediary in email answer retrieval. In: Mothe et al. [301], pp. 209–214
12.
go back to reference Alharbi, A., Stevenson, M.: Improving ranking for systematic reviews using query adaptation. In: Crestani et al. [94] Alharbi, A., Stevenson, M.: Improving ranking for systematic reviews using query adaptation. In: Crestani et al. [94]
13.
go back to reference Alkhawaldeh, R.S., Jose, J.M.: Experimental study on semi-structured peer-to-peer information retrieval network. In: Mothe et al. [301], pp. 3–14 Alkhawaldeh, R.S., Jose, J.M.: Experimental study on semi-structured peer-to-peer information retrieval network. In: Mothe et al. [301], pp. 3–14
14.
go back to reference Allan, J., et al.: Research frontiers in information retrieval - report from the third strategic workshop on information retrieval in Lorne (SWIRL 2018). In: SIGIR Forum, vol. 52, no. 1, pp. 34–90 (2018) Allan, J., et al.: Research frontiers in information retrieval - report from the third strategic workshop on information retrieval in Lorne (SWIRL 2018). In: SIGIR Forum, vol. 52, no. 1, pp. 34–90 (2018)
15.
go back to reference Amigó, E., et al.: Overview of RepLab 2013: evaluating online reputation monitoring systems. In: Forner et al. [138], pp. 333–352 Amigó, E., et al.: Overview of RepLab 2013: evaluating online reputation monitoring systems. In: Forner et al. [138], pp. 333–352
16.
go back to reference Amigó, E., et al.: Overview of RepLab 2014: author profiling and reputation dimensions for online reputation management. In: Kanoulas et al. [216], pp. 307–322 Amigó, E., et al.: Overview of RepLab 2014: author profiling and reputation dimensions for online reputation management. In: Kanoulas et al. [216], pp. 307–322
17.
go back to reference Amigó, E., Artiles, J., Gonzalo, J., Spina, D., Liu, B., Corujo, A.: WePS3 evaluation campaign: overview of the on-line reputation management task. In: Braschler et al. [61] Amigó, E., Artiles, J., Gonzalo, J., Spina, D., Liu, B., Corujo, A.: WePS3 evaluation campaign: overview of the on-line reputation management task. In: Braschler et al. [61]
18.
go back to reference Amigó, E., Corujo, A., Gonzalo, J., Meij, E., de Rijke, M.: Overview of RepLab 2012: evaluating online reputation management systems. In: Forner et al. [137] Amigó, E., Corujo, A., Gonzalo, J., Meij, E., de Rijke, M.: Overview of RepLab 2012: evaluating online reputation management systems. In: Forner et al. [137]
19.
go back to reference Amigó, E., Gonzalo, J., Verdejo, M.F.: A comparison of evaluation metrics for document filtering. In: Forner et al. [136], pp. 38–49 Amigó, E., Gonzalo, J., Verdejo, M.F.: A comparison of evaluation metrics for document filtering. In: Forner et al. [136], pp. 38–49
20.
go back to reference Anderka, M., Stein, B.: Overview of the 1st international competition on quality flaw prediction in Wikipedia. In: Forner et al. [137] Anderka, M., Stein, B.: Overview of the 1st international competition on quality flaw prediction in Wikipedia. In: Forner et al. [137]
21.
go back to reference Andrearczyk, V., Müller, H.: Deep multimodal classification of image types in biomedical journal figures. In: Bellot et al. [42], pp. 3–14 Andrearczyk, V., Müller, H.: Deep multimodal classification of image types in biomedical journal figures. In: Bellot et al. [42], pp. 3–14
22.
go back to reference Angelini, M., et al.: Cumulated relative position: a metric for ranking evaluation. In: Catarci et al. [75], pp. 112–123 Angelini, M., et al.: Cumulated relative position: a metric for ranking evaluation. In: Catarci et al. [75], pp. 112–123
23.
go back to reference Angelini, M., et al.: Measuring and analyzing the scholarly impact of experimental evaluation initiatives. In: Agosti, M., Catarci, T., Esposito, F. (eds.) Proceedings of the 10th Italian Research Conference on Digital Libraries (IRCDL 2014), vol. 38, pp. 133–137. Procedia Computer Science (2014) Angelini, M., et al.: Measuring and analyzing the scholarly impact of experimental evaluation initiatives. In: Agosti, M., Catarci, T., Esposito, F. (eds.) Proceedings of the 10th Italian Research Conference on Digital Libraries (IRCDL 2014), vol. 38, pp. 133–137. Procedia Computer Science (2014)
24.
go back to reference Angelini, M., Ferro, N., Santucci, G., Silvello, G.: Improving ranking evaluation employing visual analytics. In: Forner et al. [138], pp. 29–40 Angelini, M., Ferro, N., Santucci, G., Silvello, G.: Improving ranking evaluation employing visual analytics. In: Forner et al. [138], pp. 29–40
25.
go back to reference Antunes, H., Lopes, C.: Analyzing the adequacy of readability indicators to a non-English language. In: Crestani et al. [94] Antunes, H., Lopes, C.: Analyzing the adequacy of readability indicators to a non-English language. In: Crestani et al. [94]
26.
go back to reference Argamon, S., Juola, P.: Overview of the international authorship identification competition at PAN-2011. In: Petras et al. [373] Argamon, S., Juola, P.: Overview of the international authorship identification competition at PAN-2011. In: Petras et al. [373]
27.
go back to reference Armstrong, T.G., Moffat, A., Webber, W., Zobel, J.: Improvements that don’t add up: ad-hoc retrieval results since 1998. In: Cheung, D.W.L., Song, I.Y., Chu, W.W., Hu, X., Lin, J.J. (eds.) Proceedings of the 18th International Conference on Information and Knowledge Management (CIKM 2009), pp. 601–610. ACM Press, New York (2009) Armstrong, T.G., Moffat, A., Webber, W., Zobel, J.: Improvements that don’t add up: ad-hoc retrieval results since 1998. In: Cheung, D.W.L., Song, I.Y., Chu, W.W., Hu, X., Lin, J.J. (eds.) Proceedings of the 18th International Conference on Information and Knowledge Management (CIKM 2009), pp. 601–610. ACM Press, New York (2009)
28.
go back to reference Arni, T., Clough, P., Sanderson, M., Grubinger, M.: Overview of the ImageCLEFphoto 2008 photographic retrieval task. In: Peters et al. [359], pp. 500–511 Arni, T., Clough, P., Sanderson, M., Grubinger, M.: Overview of the ImageCLEFphoto 2008 photographic retrieval task. In: Peters et al. [359], pp. 500–511
29.
go back to reference Arora, P., Foster, J., Jones, G.J.F.: Query expansion for sentence retrieval using pseudo relevance feedback and word embedding. In: Jones et al. [209], pp. 97–103 Arora, P., Foster, J., Jones, G.J.F.: Query expansion for sentence retrieval using pseudo relevance feedback and word embedding. In: Jones et al. [209], pp. 97–103
30.
go back to reference Artiles, J., Borthwick, A., Gonzalo, J., Sekine, S., Amigó, E.: WePS-3 evaluation campaign: overview of the web people search clustering and attribute extraction tasks. In: Braschler et al. [61] Artiles, J., Borthwick, A., Gonzalo, J., Sekine, S., Amigó, E.: WePS-3 evaluation campaign: overview of the web people search clustering and attribute extraction tasks. In: Braschler et al. [61]
31.
go back to reference Azarbonyad, H., Marx, M.: How many labels? Determining the number of labels in multi-label text classification. In: Crestani et al. [94] Azarbonyad, H., Marx, M.: How many labels? Determining the number of labels in multi-label text classification. In: Crestani et al. [94]
32.
go back to reference Azarbonyad, H., Saan, F., Dehghani, D., Marx, M., Kamps, J.: Are topically diverse documents also interesting? In: Mothe et al. [301], pp. 215–221 Azarbonyad, H., Saan, F., Dehghani, D., Marx, M., Kamps, J.: Are topically diverse documents also interesting? In: Mothe et al. [301], pp. 215–221
33.
go back to reference Azarbonyad, H., Shakery, A., Faili, H.: Exploiting multiple translation resources for English-Persian cross language information retrieval. In: Forner et al. [138], pp. 93–99 Azarbonyad, H., Shakery, A., Faili, H.: Exploiting multiple translation resources for English-Persian cross language information retrieval. In: Forner et al. [138], pp. 93–99
34.
go back to reference Azzopardi, L., Balog, K.: Towards a living lab for information retrieval research and development - a proposal for a living lab for product search tasks. In: Forner et al. [136], pp. 26–37 Azzopardi, L., Balog, K.: Towards a living lab for information retrieval research and development - a proposal for a living lab for product search tasks. In: Forner et al. [136], pp. 26–37
35.
go back to reference Balog, K., Azzopardi, L., Kamps, J., de Rijke, M.: Overview of WebCLEF 2006. In: Peters et al. [357], pp. 803–819 Balog, K., Azzopardi, L., Kamps, J., de Rijke, M.: Overview of WebCLEF 2006. In: Peters et al. [357], pp. 803–819
37.
go back to reference Baradaran Hashemi, H., Shakery, A., Feili, H.: Creating a Persian-English comparable corpus. In: Agosti et al. [9], pp. 27–39 Baradaran Hashemi, H., Shakery, A., Feili, H.: Creating a Persian-English comparable corpus. In: Agosti et al. [9], pp. 27–39
38.
go back to reference Basile, A., Dwyer, G., Medvedeva, M., Rawee, J., Haagsma, H., Nissim, M.: Simply the best: minimalist system trumps complex models in author profiling. In: Bellot et al. [42], pp. 143–156 Basile, A., Dwyer, G., Medvedeva, M., Rawee, J., Haagsma, H., Nissim, M.: Simply the best: minimalist system trumps complex models in author profiling. In: Bellot et al. [42], pp. 143–156
39.
go back to reference Baudis, P., Sedivý, J.: Modeling of the question answering task in the YodaQA system. In: Mothe et al. [301], pp. 222–228 Baudis, P., Sedivý, J.: Modeling of the question answering task in the YodaQA system. In: Mothe et al. [301], pp. 222–228
40.
go back to reference Bellot, P., et al.: Overview of INEX 2014. In: Kanoulas et al. [216], pp. 212–228 Bellot, P., et al.: Overview of INEX 2014. In: Kanoulas et al. [216], pp. 212–228
41.
go back to reference Bellot, P., et al.: Overview of INEX 2013. In: Forner et al. [138], pp. 269–281 Bellot, P., et al.: Overview of INEX 2013. In: Forner et al. [138], pp. 269–281
43.
go back to reference Beloborodov, A., Braslavski, P., Driker, M.: Towards automatic evaluation of health-related CQA data. In: Kanoulas et al. [216], pp. 7–18 Beloborodov, A., Braslavski, P., Driker, M.: Towards automatic evaluation of health-related CQA data. In: Kanoulas et al. [216], pp. 7–18
44.
go back to reference Bensalem, I., Rosso, P., Chikhi, S.: A new corpus for the evaluation of Arabic intrinsic plagiarism detection. In: Forner et al. [138], pp. 53–58 Bensalem, I., Rosso, P., Chikhi, S.: A new corpus for the evaluation of Arabic intrinsic plagiarism detection. In: Forner et al. [138], pp. 53–58
45.
go back to reference Berendsen, R., Tsagkias, M., de Rijke, M., Meij, E.: Generating pseudo test collections for learning to rank scientific articles. In: Catarci et al. [75], pp. 42–53 Berendsen, R., Tsagkias, M., de Rijke, M., Meij, E.: Generating pseudo test collections for learning to rank scientific articles. In: Catarci et al. [75], pp. 42–53
46.
go back to reference Berlanga Llavori, R., Jimeno-Yepes, A., Pérez Catalán, M., Rebholz-Schuhmann, D.: Context-dependent semantic annotation in cross-lingual biomedical resources. In: Forner et al. [138], pp. 120–123 Berlanga Llavori, R., Jimeno-Yepes, A., Pérez Catalán, M., Rebholz-Schuhmann, D.: Context-dependent semantic annotation in cross-lingual biomedical resources. In: Forner et al. [138], pp. 120–123
47.
go back to reference Berlanga Llavori, R., Pérez Catalán, M., Museros Cabedo, L., Forcada, R.: Semantic discovery of resources in cloud-based PACS/RIS systems. In: Forner et al. [138], pp. 167–178 Berlanga Llavori, R., Pérez Catalán, M., Museros Cabedo, L., Forcada, R.: Semantic discovery of resources in cloud-based PACS/RIS systems. In: Forner et al. [138], pp. 167–178
48.
go back to reference Besançon, R., Chaudiron, S., Mostefa, D., Hamon, O., Timimi, I., Choukri, K.: Overview of CLEF 2008 INFILE pilot track. In: Peters et al. [359], pp. 939–946 Besançon, R., Chaudiron, S., Mostefa, D., Hamon, O., Timimi, I., Choukri, K.: Overview of CLEF 2008 INFILE pilot track. In: Peters et al. [359], pp. 939–946
49.
go back to reference Besançon, R., Chaudiron, S., Mostefa, D., Timimi, I., Choukri, K., Laïb, M.: Information filtering evaluation: overview of CLEF 2009 INFILE track. In: Peters et al. [360], pp. 342–353 Besançon, R., Chaudiron, S., Mostefa, D., Timimi, I., Choukri, K., Laïb, M.: Information filtering evaluation: overview of CLEF 2009 INFILE track. In: Peters et al. [360], pp. 342–353
50.
go back to reference Bhaskar, P., Bandyopadhyay, S.: Language independent query focused snippet generation. In: Catarci et al. [75], pp. 138–140CrossRef Bhaskar, P., Bandyopadhyay, S.: Language independent query focused snippet generation. In: Catarci et al. [75], pp. 138–140CrossRef
51.
go back to reference Blinov, V., Mishchenko, K., Bolotova, V., Braslavski, P.: A pinch of humor for short-text conversation: an information retrieval approach. In: Jones et al. [209], pp. 3–15 Blinov, V., Mishchenko, K., Bolotova, V., Braslavski, P.: A pinch of humor for short-text conversation: an information retrieval approach. In: Jones et al. [209], pp. 3–15
52.
go back to reference Bordea, G., Thiessard, F., Hamon, T., Mougin, F.: Automatic query selection for acquisition and discovery of food-drug interactions. In: Bellot et al. [42], pp. 115–120 Bordea, G., Thiessard, F., Hamon, T., Mougin, F.: Automatic query selection for acquisition and discovery of food-drug interactions. In: Bellot et al. [42], pp. 115–120
56.
go back to reference Braschler, M.: CLEF 2000 - overview of results. In: Peters [345], pp. 89–101CrossRef Braschler, M.: CLEF 2000 - overview of results. In: Peters [345], pp. 89–101CrossRef
57.
go back to reference Braschler, M.: CLEF 2001 - overview of results. In: Peters et al. [354], pp. 9–26 Braschler, M.: CLEF 2001 - overview of results. In: Peters et al. [354], pp. 9–26
58.
go back to reference Braschler, M.: CLEF 2002 - overview of results. In: Peters et al. [355], pp. 9–27CrossRef Braschler, M.: CLEF 2002 - overview of results. In: Peters et al. [355], pp. 9–27CrossRef
59.
go back to reference Braschler, M.: CLEF 2003 - overview of results. In: Peters et al. [356], pp. 44–63 Braschler, M.: CLEF 2003 - overview of results. In: Peters et al. [356], pp. 44–63
60.
go back to reference Braschler, M., Di Nunzio, G.M., Ferro, N., Peters, C.: CLEF 2004: ad hoc track overview and results analysis. In: Peters et al. [358], pp. 10–26 Braschler, M., Di Nunzio, G.M., Ferro, N., Peters, C.: CLEF 2004: ad hoc track overview and results analysis. In: Peters et al. [358], pp. 10–26
62.
go back to reference Braschler, M., Peters, C.: Cross-language evaluation forum: objectives, results, achievements. Inf. Retrieval 7(1–2), 7–31 (2004)CrossRef Braschler, M., Peters, C.: Cross-language evaluation forum: objectives, results, achievements. Inf. Retrieval 7(1–2), 7–31 (2004)CrossRef
63.
go back to reference Braslavski, P., Verberne, S., Talipov, R.: Show me how to tie a tie: evaluation of cross-lingual video retrieval. In: Fuhr et al. [143], pp. 3–15 Braslavski, P., Verberne, S., Talipov, R.: Show me how to tie a tie: evaluation of cross-lingual video retrieval. In: Fuhr et al. [143], pp. 3–15
64.
go back to reference Budíková, P., Batko, M., Botorek, J., Zezula, P.: Search-based image annotation: extracting semantics from similar images. In: Mothe et al. [301], pp. 327–339 Budíková, P., Batko, M., Botorek, J., Zezula, P.: Search-based image annotation: extracting semantics from similar images. In: Mothe et al. [301], pp. 327–339
65.
go back to reference Buraya, K., Farseev, A., Filchenkov, A.: Multi-view personality profiling based on longitudinal data. In: Bellot et al. [42], pp. 15–27 Buraya, K., Farseev, A., Filchenkov, A.: Multi-view personality profiling based on longitudinal data. In: Bellot et al. [42], pp. 15–27
66.
go back to reference Cabanac, G., Hubert, G., Boughanem, M., Chrisment, C.: Tie-breaking bias: effect of an uncontrolled parameter on information retrieval evaluation. In: Agosti et al. [9], pp. 112–123 Cabanac, G., Hubert, G., Boughanem, M., Chrisment, C.: Tie-breaking bias: effect of an uncontrolled parameter on information retrieval evaluation. In: Agosti et al. [9], pp. 112–123
72.
go back to reference Caputo, B., et al.: ImageCLEF 2014: overview and analysis of the results. In: Kanoulas et al. [216], pp. 192–211 Caputo, B., et al.: ImageCLEF 2014: overview and analysis of the results. In: Kanoulas et al. [216], pp. 192–211
73.
go back to reference Caputo, B., et al.: ImageCLEF 2013: the vision, the data and the open challenges. In: Forner et al. [138], pp. 250–268 Caputo, B., et al.: ImageCLEF 2013: the vision, the data and the open challenges. In: Forner et al. [138], pp. 250–268
74.
go back to reference Cassidy, T., Ji, H., Deng, H., Zheng, J., Han, J.: Analysis and refinement of cross-lingual entity linking. In: Catarci et al. [75], pp. 1–12 Cassidy, T., Ji, H., Deng, H., Zheng, J., Han, J.: Analysis and refinement of cross-lingual entity linking. In: Catarci et al. [75], pp. 1–12
76.
go back to reference Chaa, M., Nouali, O., Bellot, P.: Combining tags and reviews to improve social book search performance. In: Bellot et al. [42], pp. 64–75 Chaa, M., Nouali, O., Bellot, P.: Combining tags and reviews to improve social book search performance. In: Bellot et al. [42], pp. 64–75
77.
go back to reference Chappell, T., Geva, S.: Overview of the INEX 2012 relevance feedback track. In: Forner et al. [137] Chappell, T., Geva, S.: Overview of the INEX 2012 relevance feedback track. In: Forner et al. [137]
78.
go back to reference Chidlovskii, B., Csurka, G., Clinchant, S.: Evaluating stacked marginalised denoising autoencoders within domain adaptation methods. In: Mothe et al. [301], pp. 15–27 Chidlovskii, B., Csurka, G., Clinchant, S.: Evaluating stacked marginalised denoising autoencoders within domain adaptation methods. In: Mothe et al. [301], pp. 15–27
79.
go back to reference Chikka, V.R., Mariyasagayam, N., Niwa, Y., Karlapalem, K.: Information extraction from clinical documents: towards disease/disorder template filling. In: Mothe et al. [301], pp. 389–401 Chikka, V.R., Mariyasagayam, N., Niwa, Y., Karlapalem, K.: Information extraction from clinical documents: towards disease/disorder template filling. In: Mothe et al. [301], pp. 389–401
80.
go back to reference Chuklin, A., Severyn, A., Trippas, J., Alfonseca, E., Silen, H., Spina, D.: Using audio transformations to improve comprehension in voice question answering. In: Crestani et al. [94] Chuklin, A., Severyn, A., Trippas, J., Alfonseca, E., Silen, H., Spina, D.: Using audio transformations to improve comprehension in voice question answering. In: Crestani et al. [94]
81.
go back to reference Cimiano, P., Lopez, V., Unger, C., Cabrio, E., Ngonga Ngomo, A.C., Walter, S.: Multilingual question answering over linked data (QALD-3): lab overview. In: Forner et al. [138], pp. 321–332 Cimiano, P., Lopez, V., Unger, C., Cabrio, E., Ngonga Ngomo, A.C., Walter, S.: Multilingual question answering over linked data (QALD-3): lab overview. In: Forner et al. [138], pp. 321–332
82.
go back to reference Clough, P., Goodale, P.: Selecting success criteria: experiences with an academic library catalogue. In: Forner et al. [138], pp. 59–70 Clough, P., Goodale, P.: Selecting success criteria: experiences with an academic library catalogue. In: Forner et al. [138], pp. 59–70
83.
go back to reference Clough, P., Grubinger, M., Deselaers, T., Hanbury, A., Müller, H.: Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks. In: Peters et al. [357], pp. 223–256 Clough, P., Grubinger, M., Deselaers, T., Hanbury, A., Müller, H.: Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks. In: Peters et al. [357], pp. 223–256
84.
go back to reference Clough, P., et al.: The CLEF 2005 cross-language image retrieval track. In: Peters et al. [365], pp. 535–557 Clough, P., et al.: The CLEF 2005 cross-language image retrieval track. In: Peters et al. [365], pp. 535–557
85.
go back to reference Clough, P., Müller, H., Sanderson, M.: The CLEF 2004 cross-language image retrieval track. In: Peters et al. [358], pp. 597–613 Clough, P., Müller, H., Sanderson, M.: The CLEF 2004 cross-language image retrieval track. In: Peters et al. [358], pp. 597–613
86.
go back to reference Clough, P., Sanderson, M.: The CLEF 2003 cross language image retrieval track. In: Peters et al. [356], pp. 581–593 Clough, P., Sanderson, M.: The CLEF 2003 cross language image retrieval track. In: Peters et al. [356], pp. 581–593
87.
go back to reference Clough, P., Tsikrika, T.: Multi-lingual retrieval of pictures in ImageCLEF. In: Ferro and Peters [130] Clough, P., Tsikrika, T.: Multi-lingual retrieval of pictures in ImageCLEF. In: Ferro and Peters [130]
88.
go back to reference Clough, P., Willett, P., Lim, J.: Unfair means: use cases beyond plagiarism. In: Mothe et al. [301], pp. 229–234 Clough, P., Willett, P., Lim, J.: Unfair means: use cases beyond plagiarism. In: Mothe et al. [301], pp. 229–234
89.
go back to reference Coello-Guilarte, D.L., Ortega Mendoza, R.M., Villaseñor-Pineda, L., Montes-y Gómez, M.: Cross-lingual depression detection in Twitter using bilingual word-level alignment. In: Crestani et al. [94] Coello-Guilarte, D.L., Ortega Mendoza, R.M., Villaseñor-Pineda, L., Montes-y Gómez, M.: Cross-lingual depression detection in Twitter using bilingual word-level alignment. In: Crestani et al. [94]
90.
go back to reference Collovini, S., Marcelo de Bairros Filho, P., Vieira, R.: Analysing the role of representation choices in portuguese relation extraction. In: Mothe et al. [301], pp. 105–116 Collovini, S., Marcelo de Bairros Filho, P., Vieira, R.: Analysing the role of representation choices in portuguese relation extraction. In: Mothe et al. [301], pp. 105–116
91.
go back to reference Conlan, O., Fraser, K., Kelly, L., Yousuf, B.: A user modeling shared challenge proposal. In: Crestani et al. [94] Conlan, O., Fraser, K., Kelly, L., Yousuf, B.: A user modeling shared challenge proposal. In: Crestani et al. [94]
92.
go back to reference Corezola Pereira, R., Pereira Moreira, V., Galante, R.: A new approach for cross-language plagiarism analysis. In: Agosti et al. [9], pp. 15–26CrossRef Corezola Pereira, R., Pereira Moreira, V., Galante, R.: A new approach for cross-language plagiarism analysis. In: Agosti et al. [9], pp. 15–26CrossRef
93.
go back to reference Cossu, J.V., Ferreira, E., Janod, K., Gaillard, J., El-Bèze, M.: NLP-based classifiers to generalize expert assessments in e-reputation. In: Mothe et al. [301], pp. 340–351 Cossu, J.V., Ferreira, E., Janod, K., Gaillard, J., El-Bèze, M.: NLP-based classifiers to generalize expert assessments in e-reputation. In: Mothe et al. [301], pp. 340–351
94.
go back to reference Crestani, F., et al. (eds.): CLEF 2019. LNCS, vol. 11696. Springer, Heidelberg (2019) Crestani, F., et al. (eds.): CLEF 2019. LNCS, vol. 11696. Springer, Heidelberg (2019)
95.
go back to reference Custódio, J., Paraboni, I.: An ensemble approach to cross-domain authorship attribution. In: Crestani et al. [94] Custódio, J., Paraboni, I.: An ensemble approach to cross-domain authorship attribution. In: Crestani et al. [94]
96.
go back to reference Dadashkarimi, J., Esfahani, H.N., Faili, H., Shakery, A.: SS4MCT: a statistical stemmer for morphologically complex texts. In: Fuhr et al. [143], pp. 201–207 Dadashkarimi, J., Esfahani, H.N., Faili, H., Shakery, A.: SS4MCT: a statistical stemmer for morphologically complex texts. In: Fuhr et al. [143], pp. 201–207
97.
go back to reference Daelemans, W., et al.: Overview of PAN 2019: bots and gender profiling, celebrity profiling, cross-domain authorship attribution and style change detection. In: Crestani et al. [94] Daelemans, W., et al.: Overview of PAN 2019: bots and gender profiling, celebrity profiling, cross-domain authorship attribution and style change detection. In: Crestani et al. [94]
98.
go back to reference Dehghani, M., Azarbonyad, H., Kamps, J., Marx, M.: Two-way parsimonious classification models for evolving hierarchies. In: Fuhr et al. [143], pp. 69–82 Dehghani, M., Azarbonyad, H., Kamps, J., Marx, M.: Two-way parsimonious classification models for evolving hierarchies. In: Fuhr et al. [143], pp. 69–82
99.
go back to reference Déjean, S., Mothe, J., Ullah, M.Z.: Studying the variability of system setting effectiveness by data analytics and visualization. In: Crestani et al. [94] Déjean, S., Mothe, J., Ullah, M.Z.: Studying the variability of system setting effectiveness by data analytics and visualization. In: Crestani et al. [94]
100.
go back to reference Deneu, B., Servajean, M., Botella, C., Joly, A.: Evaluation of deep species distribution models using environment and co-occurrences. In: Crestani et al. [94] Deneu, B., Servajean, M., Botella, C., Joly, A.: Evaluation of deep species distribution models using environment and co-occurrences. In: Crestani et al. [94]
101.
go back to reference Deselaers, T., Deserno, T.M.: Medical image annotation in ImageCLEF 2008. In: Peters et al. [359], pp. 523–530 Deselaers, T., Deserno, T.M.: Medical image annotation in ImageCLEF 2008. In: Peters et al. [359], pp. 523–530
102.
go back to reference Deselaers, T., Hanbury, A.: The visual concept detection task in ImageCLEF 2008. In: Peters et al. [359], pp. 531–538 Deselaers, T., Hanbury, A.: The visual concept detection task in ImageCLEF 2008. In: Peters et al. [359], pp. 531–538
103.
go back to reference Deselaers, T., et al.: Overview of the ImageCLEF 2007 object retrieval task. In: Peters et al. [366], pp. 445–471 Deselaers, T., et al.: Overview of the ImageCLEF 2007 object retrieval task. In: Peters et al. [366], pp. 445–471
104.
go back to reference Devezas, J., Nunes, S.: Index-based semantic tagging for efficient query interpretation. In: Fuhr et al. [143], pp. 208–213 Devezas, J., Nunes, S.: Index-based semantic tagging for efficient query interpretation. In: Fuhr et al. [143], pp. 208–213
105.
go back to reference Di Buccio, E., Dussin, M., Ferro, N., Masiero, I., Santucci, G., Tino, G.: To Re-rank or to re-query: can visual analytics solve this dilemma? In: Forner et al. [136], pp. 119–130 Di Buccio, E., Dussin, M., Ferro, N., Masiero, I., Santucci, G., Tino, G.: To Re-rank or to re-query: can visual analytics solve this dilemma? In: Forner et al. [136], pp. 119–130
106.
go back to reference Di Nunzio, G.M., Ferro, N., Jones, G.J.F., Peters, C.: CLEF 2005: ad hoc track overview. In: Peters et al. [365], pp. 11–36 Di Nunzio, G.M., Ferro, N., Jones, G.J.F., Peters, C.: CLEF 2005: ad hoc track overview. In: Peters et al. [365], pp. 11–36
107.
go back to reference Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2006: ad hoc track overview. In: Peters et al. [357], pp. 21–34 Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2006: ad hoc track overview. In: Peters et al. [357], pp. 21–34
108.
go back to reference Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2007: ad hoc track overview. In: Peters et al. [366], pp. 13–32 Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2007: ad hoc track overview. In: Peters et al. [366], pp. 13–32
109.
go back to reference Di Nunzio, G.M., Leveling, J., Mandl, T.: LogCLEF 2011 multilingual log file analysis: language identification, query classification, and success of a query. In: Petras et al. [373] Di Nunzio, G.M., Leveling, J., Mandl, T.: LogCLEF 2011 multilingual log file analysis: language identification, query classification, and success of a query. In: Petras et al. [373]
110.
go back to reference Di Nunzio, G.M., Vezzani, F.: Using R markdown for replicable experiments in evidence based medicine. In: Bellot et al. [42], pp. 28–39 Di Nunzio, G.M., Vezzani, F.: Using R markdown for replicable experiments in evidence based medicine. In: Bellot et al. [42], pp. 28–39
111.
go back to reference Dicente Cid, Y., Batmanghelich, K., Müller, H.: Textured graph-based model of the lungs: application on tuberculosis type classification and multi-drug resistance detection. In: Bellot et al. [42], pp. 157–168 Dicente Cid, Y., Batmanghelich, K., Müller, H.: Textured graph-based model of the lungs: application on tuberculosis type classification and multi-drug resistance detection. In: Bellot et al. [42], pp. 157–168
112.
go back to reference Dietz, F., Petras, V.: A component-level analysis of an academic search test collection. Part I: system and collection configurations. In: Jones et al. [209], pp. 16–28 Dietz, F., Petras, V.: A component-level analysis of an academic search test collection. Part I: system and collection configurations. In: Jones et al. [209], pp. 16–28
113.
go back to reference Dietz, F., Petras, V.: A component-level analysis of an academic search test collection.- Part II: query analysis. In: Jones et al. [209], pp. 29–42 Dietz, F., Petras, V.: A component-level analysis of an academic search test collection.- Part II: query analysis. In: Jones et al. [209], pp. 29–42
114.
go back to reference Domann, J., Lommatzsch, A.: A highly available real-time news recommender based on apache spark. In: Jones et al. [209], pp. 161–172 Domann, J., Lommatzsch, A.: A highly available real-time news recommender based on apache spark. In: Jones et al. [209], pp. 161–172
115.
go back to reference Elsayed, T., et al.: Overview of the CLEF-2019 CheckThat!: automatic identification and verification of claims. In: Crestani et al. [94] Elsayed, T., et al.: Overview of the CLEF-2019 CheckThat!: automatic identification and verification of claims. In: Crestani et al. [94]
116.
go back to reference Ermakova, K.: A method for short message contextualization: experiments at CLEF/INEX. In: Mothe et al. [301], pp. 352–363 Ermakova, K.: A method for short message contextualization: experiments at CLEF/INEX. In: Mothe et al. [301], pp. 352–363
117.
go back to reference Ermakova, L., Goeuriot, L., Mothe, J., Mulhem, P., Nie, J.Y., SanJuan, E.: CLEF 2017 microblog cultural contextualization lab overview. In: Jones et al. [209], pp. 304–314 Ermakova, L., Goeuriot, L., Mothe, J., Mulhem, P., Nie, J.Y., SanJuan, E.: CLEF 2017 microblog cultural contextualization lab overview. In: Jones et al. [209], pp. 304–314
118.
go back to reference Esuli, F., Sebastiani, F.: Evaluating information extraction. In: Agosti et al. [9], pp. 100–111CrossRef Esuli, F., Sebastiani, F.: Evaluating information extraction. In: Agosti et al. [9], pp. 100–111CrossRef
119.
go back to reference Ezzeldin, A.M., Kholief, M.H., El-Sonbaty, Y.: ALQASIM: Arabic language question answer selection in machines. In: Forner et al. [138], pp. 100–103 Ezzeldin, A.M., Kholief, M.H., El-Sonbaty, Y.: ALQASIM: Arabic language question answer selection in machines. In: Forner et al. [138], pp. 100–103
120.
go back to reference Federico, M., Bertoldi, N., Levow, G.A., Jones, G.J.F.: CLEF 2004 cross-language spoken document retrieval track. In: Peters et al. [358], pp. 816–820 Federico, M., Bertoldi, N., Levow, G.A., Jones, G.J.F.: CLEF 2004 cross-language spoken document retrieval track. In: Peters et al. [358], pp. 816–820
121.
go back to reference Federico, M., Jones, G.J.F.: The CLEF 2003 cross-language spoken document retrieval track. In: Peters et al. [356], p. 646 Federico, M., Jones, G.J.F.: The CLEF 2003 cross-language spoken document retrieval track. In: Peters et al. [356], p. 646
122.
go back to reference Ferrante, M., Ferro, N., Maistro, M.: Rethinking how to extend average precision to graded relevance. In: Kanoulas et al. [216], pp. 19–30 Ferrante, M., Ferro, N., Maistro, M.: Rethinking how to extend average precision to graded relevance. In: Kanoulas et al. [216], pp. 19–30
123.
go back to reference Ferro, N.: What happened in CLEF... for a while? In: Crestani et al. [94] Ferro, N.: What happened in CLEF... for a while? In: Crestani et al. [94]
124.
go back to reference Ferro, N., et al.: Manifesto from Dagstuhl Perspectives Workshop 17442 - From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences, vol. 7, no. 1, pp. 96–139. Dagstuhl Manifestos, Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Germany (2018) Ferro, N., et al.: Manifesto from Dagstuhl Perspectives Workshop 17442 - From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences, vol. 7, no. 1, pp. 96–139. Dagstuhl Manifestos, Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Germany (2018)
126.
go back to reference Ferro, N., Harman, D.: CLEF 2009: Grid@CLEF pilot track overview. In: Peters et al. [360], pp. 552–565 Ferro, N., Harman, D.: CLEF 2009: Grid@CLEF pilot track overview. In: Peters et al. [360], pp. 552–565
127.
go back to reference Ferro, N., Maistro, M., Sakai, T., Soboroff, I.: CENTRE@CLEF2018: overview of the replicability task. In: Cappellato et al. [71] Ferro, N., Maistro, M., Sakai, T., Soboroff, I.: CENTRE@CLEF2018: overview of the replicability task. In: Cappellato et al. [71]
128.
go back to reference Ferro, N., Peters, C.: CLEF 2009 ad hoc track overview: TEL & Persian tasks. In: Peters et al. [360], pp. 13–35 Ferro, N., Peters, C.: CLEF 2009 ad hoc track overview: TEL & Persian tasks. In: Peters et al. [360], pp. 13–35
129.
go back to reference Ferro, N., Peters, C.: From multilingual to multimodal: the evolution of CLEF over two decades. In: Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF [130] Ferro, N., Peters, C.: From multilingual to multimodal: the evolution of CLEF over two decades. In: Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF [130]
131.
go back to reference Ferro, N., Santucci, G.: Visual analytics and IR experimental evaluation. In: Ferro and Peters [130] Ferro, N., Santucci, G.: Visual analytics and IR experimental evaluation. In: Ferro and Peters [130]
132.
go back to reference Ferro, N., Silvello, G.: CLEF 15th birthday: what can we learn from ad hoc retrieval? In: Kanoulas et al. [216], pp. 31–43 Ferro, N., Silvello, G.: CLEF 15th birthday: what can we learn from ad hoc retrieval? In: Kanoulas et al. [216], pp. 31–43
133.
go back to reference Ferro, N., Silvello, G.: The CLEF monolingual grid of points. In: Fuhr et al. [143], pp. 16–27 Ferro, N., Silvello, G.: The CLEF monolingual grid of points. In: Fuhr et al. [143], pp. 16–27
134.
go back to reference Ferro, N., Silvello, G.: 3.5K runs, 5K topics, 3M assessments and 70M measures: what trends in 10 years of Adhoc-ish CLEF? Inf. Process. Manag. 53(1), 175–202 (2017)CrossRef Ferro, N., Silvello, G.: 3.5K runs, 5K topics, 3M assessments and 70M measures: what trends in 10 years of Adhoc-ish CLEF? Inf. Process. Manag. 53(1), 175–202 (2017)CrossRef
135.
go back to reference Fontanella, S., Rodríguez-Sánchez, A.J., Piater, J., Szedmak, S.: Kronecker decomposition for image classification. In: Fuhr et al. [143], pp. 137–149 Fontanella, S., Rodríguez-Sánchez, A.J., Piater, J., Szedmak, S.: Kronecker decomposition for image classification. In: Fuhr et al. [143], pp. 137–149
140.
go back to reference Forner, P., et al.: Overview of the Clef 2008 Multilingual Question Answering Track. In: Peters et al. [359], pp. 262–295 Forner, P., et al.: Overview of the Clef 2008 Multilingual Question Answering Track. In: Peters et al. [359], pp. 262–295
141.
go back to reference Franco-Salvador, M., Rangel Pardo, F.M., Rosso, P., Taulé, M., Martí, M.A.: Language variety identification using distributed representations of words and documents. In: Mothe et al. [301], pp. 28–40 Franco-Salvador, M., Rangel Pardo, F.M., Rosso, P., Taulé, M., Martí, M.A.: Language variety identification using distributed representations of words and documents. In: Mothe et al. [301], pp. 28–40
142.
go back to reference Fuhr, N.: Reproducibility and validity in CLEF. In: Ferro and Peters [130] Fuhr, N.: Reproducibility and validity in CLEF. In: Ferro and Peters [130]
144.
go back to reference Gäde, M., Ferro, N., Lestari Paramita, M.: CHiC 2011 - cultural heritage in CLEF: from use cases to evaluation in practice for multilingual information access to cultural heritage. In: Petras et al. [373] Gäde, M., Ferro, N., Lestari Paramita, M.: CHiC 2011 - cultural heritage in CLEF: from use cases to evaluation in practice for multilingual information access to cultural heritage. In: Petras et al. [373]
145.
go back to reference Gäde, M., Stiller, J., Petras, V.: Which log for which information? Gathering multilingual data from different log file types. In: Agosti et al. [9], pp. 70–81 Gäde, M., Stiller, J., Petras, V.: Which log for which information? Gathering multilingual data from different log file types. In: Agosti et al. [9], pp. 70–81
146.
go back to reference Galuscáková, P., Pecina, P., Hajic, J.: Penalty functions for evaluation measures of unsegmented speech retrieval. In: Catarci et al. [75], pp. 100–111 Galuscáková, P., Pecina, P., Hajic, J.: Penalty functions for evaluation measures of unsegmented speech retrieval. In: Catarci et al. [75], pp. 100–111
147.
go back to reference Ganguly, D., Jones, G.J.F.: A gamified approach to relevance judgement. In: Fuhr et al. [143], pp. 214–220 Ganguly, D., Jones, G.J.F.: A gamified approach to relevance judgement. In: Fuhr et al. [143], pp. 214–220
148.
go back to reference Ganguly, D., Leveling, J., Jones, G.J.F.: Simulation of within-session query variations using a text segmentation approach. In: Forner et al. [136], pp. 89–94 Ganguly, D., Leveling, J., Jones, G.J.F.: Simulation of within-session query variations using a text segmentation approach. In: Forner et al. [136], pp. 89–94
149.
go back to reference Ganguly, D., Leveling, J., Jones, G.J.F.: A case study in decompounding for Bengali information retrieval. In: Forner et al. [138], pp. 108–119 Ganguly, D., Leveling, J., Jones, G.J.F.: A case study in decompounding for Bengali information retrieval. In: Forner et al. [138], pp. 108–119
150.
go back to reference Gebremeskel, G.G., de Vries, A.P.: Random performance differences between online recommender system algorithms. In: Fuhr et al. [143], pp. 187–200 Gebremeskel, G.G., de Vries, A.P.: Random performance differences between online recommender system algorithms. In: Fuhr et al. [143], pp. 187–200
151.
go back to reference Gey, F., et al.: GeoCLEF 2006: the CLEF 2006 cross-language geographic information retrieval track overview. In: Peters et al. [357], pp. 852–876 Gey, F., et al.: GeoCLEF 2006: the CLEF 2006 cross-language geographic information retrieval track overview. In: Peters et al. [357], pp. 852–876
152.
go back to reference Gey, F.C., Larson, R.R., Sanderson, M., Joho, H., Clough, P., Petras, V.: GeoCLEF: the CLEF 2005 cross-language geographic information retrieval track overview. In: Peters et al. [365], pp. 908–919 Gey, F.C., Larson, R.R., Sanderson, M., Joho, H., Clough, P., Petras, V.: GeoCLEF: the CLEF 2005 cross-language geographic information retrieval track overview. In: Peters et al. [365], pp. 908–919
153.
go back to reference Ghosh, S., Singhania, P., Singh, S., Rudra, K., Ghosh, S.: Stance detection in web and social media: a comparative study. In: Crestani et al. [94] Ghosh, S., Singhania, P., Singh, S., Rudra, K., Ghosh, S.: Stance detection in web and social media: a comparative study. In: Crestani et al. [94]
154.
go back to reference Giampiccolo, D., et al.: Overview of the CLEF 2007 multilingual question answering track. In: Peters et al. [366], pp. 200–236 Giampiccolo, D., et al.: Overview of the CLEF 2007 multilingual question answering track. In: Peters et al. [366], pp. 200–236
155.
go back to reference Gînsca, A.L., Popescu, A., Lupu, M., Iftene, A., Kanellos, I.: Evaluating user image tagging credibility. In: Mothe et al. [301], pp. 41–52 Gînsca, A.L., Popescu, A., Lupu, M., Iftene, A., Kanellos, I.: Evaluating user image tagging credibility. In: Mothe et al. [301], pp. 41–52
156.
go back to reference Glinos, D.G.: Discovering similar passages within large text documents. In: Kanoulas et al. [216], pp. 98–109 Glinos, D.G.: Discovering similar passages within large text documents. In: Kanoulas et al. [216], pp. 98–109
157.
go back to reference Gobeill, J., Gaudinat, A., Ruch, P.: Instance-based learning for tweet monitoring and categorization. In: Mothe et al. [301], pp. 235–240 Gobeill, J., Gaudinat, A., Ruch, P.: Instance-based learning for tweet monitoring and categorization. In: Mothe et al. [301], pp. 235–240
158.
go back to reference Goëau, H., et al.: The CLEF 2011 plant images classification task. In: Petras et al. [373] Goëau, H., et al.: The CLEF 2011 plant images classification task. In: Petras et al. [373]
159.
go back to reference Goëau, H., et al.: The ImageCLEF 2012 plant identification task. In: Forner et al. [137] Goëau, H., et al.: The ImageCLEF 2012 plant identification task. In: Forner et al. [137]
160.
go back to reference Goeuriot, L., et al.: Overview of the CLEF eHealth evaluation lab 2015. In: Mothe et al. [301], pp. 429–443 Goeuriot, L., et al.: Overview of the CLEF eHealth evaluation lab 2015. In: Mothe et al. [301], pp. 429–443
161.
go back to reference Goeuriot, L., et al.: CLEF 2017 eHealth evaluation lab overview. In: Jones et al. [209], pp. 291–303 Goeuriot, L., et al.: CLEF 2017 eHealth evaluation lab overview. In: Jones et al. [209], pp. 291–303
162.
go back to reference Goeuriot, L., Mothe, J., Mulhem, P., Murtagh, F., SanJuan, E.: Overview of the CLEF 2016 cultural micro-blog contextualization workshop. In: Fuhr et al. [143], pp. 371–378 Goeuriot, L., Mothe, J., Mulhem, P., Murtagh, F., SanJuan, E.: Overview of the CLEF 2016 cultural micro-blog contextualization workshop. In: Fuhr et al. [143], pp. 371–378
163.
go back to reference Gollub, T., et al.: Recent trends in digital text forensics and its evaluation - plagiarism detection, author identification, and author profiling. In: Forner et al. [138], pp. 282–302 Gollub, T., et al.: Recent trends in digital text forensics and its evaluation - plagiarism detection, author identification, and author profiling. In: Forner et al. [138], pp. 282–302
164.
go back to reference Gómez-Adorno, H., Martín-del Campo-Rodríguez, C., Sidorov, G., Alemán, Y., Vilariño, D., Pinto, D.: Hierarchical clustering analysis: the best-performing approach at PAN 2017 author clustering task. In: Bellot et al. [42], pp. 216–223 Gómez-Adorno, H., Martín-del Campo-Rodríguez, C., Sidorov, G., Alemán, Y., Vilariño, D., Pinto, D.: Hierarchical clustering analysis: the best-performing approach at PAN 2017 author clustering task. In: Bellot et al. [42], pp. 216–223
165.
go back to reference Gonzalo, J., Clough, P., Karlgren, J.: Overview of iCLEF 2008: search log analysis for multilingual image retrieval. In: Peters et al. [359], pp. 227–235 Gonzalo, J., Clough, P., Karlgren, J.: Overview of iCLEF 2008: search log analysis for multilingual image retrieval. In: Peters et al. [359], pp. 227–235
166.
go back to reference Gonzalo, J., Clough, P., Vallin, A.: Overview of the CLEF 2005 interactive track. In: Peters et al. [365], pp. 251–262 Gonzalo, J., Clough, P., Vallin, A.: Overview of the CLEF 2005 interactive track. In: Peters et al. [365], pp. 251–262
167.
go back to reference Gonzalo, J., Oard, D.W.: The CLEF 2002 interactive track. In: Peters et al. [355], pp. 372–382CrossRef Gonzalo, J., Oard, D.W.: The CLEF 2002 interactive track. In: Peters et al. [355], pp. 372–382CrossRef
168.
go back to reference Gonzalo, J., Oard, D.W.: iCLEF 2004 track overview: pilot experiments in interactive cross-language question answering. In: Peters et al. [358], pp. 310–322 Gonzalo, J., Oard, D.W.: iCLEF 2004 track overview: pilot experiments in interactive cross-language question answering. In: Peters et al. [358], pp. 310–322
169.
go back to reference Gonzalo, J., Peinado, V., Clough, P., Karlgren, J.: Overview of iCLEF 2009: exploring search behaviour in a multilingual folksonomy environment. In: Peters et al. [368], pp. 13–20 Gonzalo, J., Peinado, V., Clough, P., Karlgren, J.: Overview of iCLEF 2009: exploring search behaviour in a multilingual folksonomy environment. In: Peters et al. [368], pp. 13–20
170.
go back to reference Goodwin, T., Harabagiu, S.M.: The impact of belief values on the identification of patient cohorts. In: Forner et al. [138], pp. 155–166 Goodwin, T., Harabagiu, S.M.: The impact of belief values on the identification of patient cohorts. In: Forner et al. [138], pp. 155–166
171.
go back to reference Grotov, A., Chuklin, A., Markov, I., Stout, L., Xumara, F., de Rijke, M.: A comparative study of click models for web search. In: Mothe et al. [301], pp. 78–90 Grotov, A., Chuklin, A., Markov, I., Stout, L., Xumara, F., de Rijke, M.: A comparative study of click models for web search. In: Mothe et al. [301], pp. 78–90
172.
go back to reference Grubinger, M., Clough, P., Hanbury, A., Müller, H.: Overview of the ImageCLEFphoto 2007 photographic retrieval task. In: Peters et al. [366], pp. 433–444 Grubinger, M., Clough, P., Hanbury, A., Müller, H.: Overview of the ImageCLEFphoto 2007 photographic retrieval task. In: Peters et al. [366], pp. 433–444
173.
go back to reference Gupta, P., Barrón-Cedeño, A., Rosso, P.: Cross-language high similarity search using a conceptual thesaurus. In: Catarci et al. [75], pp. 67–75CrossRef Gupta, P., Barrón-Cedeño, A., Rosso, P.: Cross-language high similarity search using a conceptual thesaurus. In: Catarci et al. [75], pp. 67–75CrossRef
174.
go back to reference Hagen, M., Glimm, C.: Supporting more-like-this information needs: finding similar web content in different scenarios. In: Kanoulas et al. [216], pp. 50–61 Hagen, M., Glimm, C.: Supporting more-like-this information needs: finding similar web content in different scenarios. In: Kanoulas et al. [216], pp. 50–61
175.
go back to reference Hall, M., Toms, E.: Building a common framework for IIR evaluation. In: Forner et al. [138], pp. 17–28 Hall, M., Toms, E.: Building a common framework for IIR evaluation. In: Forner et al. [138], pp. 17–28
176.
go back to reference Halvani, O., Graner, L.: Rethinking the evaluation methodology of authorship verification methods. In: Bellot et al. [42], pp. 40–51 Halvani, O., Graner, L.: Rethinking the evaluation methodology of authorship verification methods. In: Bellot et al. [42], pp. 40–51
177.
go back to reference Hammarström, H.: Automatic annotation of bibliographical references for descriptive language materials. In: Forner et al. [136], pp. 62–73CrossRef Hammarström, H.: Automatic annotation of bibliographical references for descriptive language materials. In: Forner et al. [136], pp. 62–73CrossRef
178.
go back to reference Han Lee, S., Loong Chang, Y., Seng Chan, S., Alexis, J., Bonnet, P., Goëau, H.: Plant classification based on gated recurrent unit. In: Bellot et al. [42], pp. 169–180 Han Lee, S., Loong Chang, Y., Seng Chan, S., Alexis, J., Bonnet, P., Goëau, H.: Plant classification based on gated recurrent unit. In: Bellot et al. [42], pp. 169–180
179.
go back to reference Hanbury, A., Müller, H.: Automated component-level evaluation: present and future. In: Agosti et al. [9], pp. 124–135 Hanbury, A., Müller, H.: Automated component-level evaluation: present and future. In: Agosti et al. [9], pp. 124–135
180.
go back to reference Hanbury, A., Müller, H., Langs, G., Weber, M.A., Menze, B.H., Salas Fernandez, T.: Bringing the algorithms to the data: cloud-based benchmarking for medical image analysis. In: Catarci et al. [75], pp. 24–29 Hanbury, A., Müller, H., Langs, G., Weber, M.A., Menze, B.H., Salas Fernandez, T.: Bringing the algorithms to the data: cloud-based benchmarking for medical image analysis. In: Catarci et al. [75], pp. 24–29
181.
go back to reference Harman, D.K.: Information Retrieval Evaluation. Morgan & Claypool Publishers, San Rafael (2011)CrossRef Harman, D.K.: Information Retrieval Evaluation. Morgan & Claypool Publishers, San Rafael (2011)CrossRef
182.
go back to reference Harman, D.K., Voorhees, E.M. (eds.): TREC. Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge (2005) Harman, D.K., Voorhees, E.M. (eds.): TREC. Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge (2005)
183.
go back to reference Harris, C.G., Xu, T.: The importance of visual context clues in multimedia translation. In: Forner et al. [136], pp. 107–118 Harris, C.G., Xu, T.: The importance of visual context clues in multimedia translation. In: Forner et al. [136], pp. 107–118
184.
go back to reference Hasan, S.A., et al.: Attention-based medical caption generation with image modality classification and clinical concept mapping. In: Bellot et al. [42], pp. 224–230 Hasan, S.A., et al.: Attention-based medical caption generation with image modality classification and clinical concept mapping. In: Bellot et al. [42], pp. 224–230
185.
go back to reference Hiemstra, D., Hauff, C.: MapReduce for information retrieval evaluation: “let’s quickly test this on 12 TB of data”. In: Agosti et al. [9], pp. 64–69 Hiemstra, D., Hauff, C.: MapReduce for information retrieval evaluation: “let’s quickly test this on 12 TB of data”. In: Agosti et al. [9], pp. 64–69
186.
go back to reference Hoang, T.B.N., Mothe, J., Baillon, M.: TwitCID: a collection of data sets for studies on information diffusion on social networks. In: Crestani et al. [94] Hoang, T.B.N., Mothe, J., Baillon, M.: TwitCID: a collection of data sets for studies on information diffusion on social networks. In: Crestani et al. [94]
187.
go back to reference Hopfgartner, F., Balog, K., Lommatzsch, A., Kelly, L., Kille, B., Schuth, A., Larson, M.: Continuous evaluation of large-scale information access systems: a case for living labs. In: Ferro and Peters [130] Hopfgartner, F., Balog, K., Lommatzsch, A., Kelly, L., Kille, B., Schuth, A., Larson, M.: Continuous evaluation of large-scale information access systems: a case for living labs. In: Ferro and Peters [130]
188.
go back to reference Hopfgartner, F., Kille, B., Lommatzsch, A., Plumbaum, T., Brodt, T., Heintz, T.: Benchmarking news recommendations in a living lab. In: Kanoulas et al. [216], pp. 250–267 Hopfgartner, F., Kille, B., Lommatzsch, A., Plumbaum, T., Brodt, T., Heintz, T.: Benchmarking news recommendations in a living lab. In: Kanoulas et al. [216], pp. 250–267
190.
go back to reference Hürriyetoğlu, A., et al.: Overview of CLEF 2019 lab ProtestNews: extracting protests from news in a cross-context setting. In: Crestani et al. [94] Hürriyetoğlu, A., et al.: Overview of CLEF 2019 lab ProtestNews: extracting protests from news in a cross-context setting. In: Crestani et al. [94]
191.
go back to reference Huurnink, B., Hofmann, K., de Rijke, M., Bron, M.: Validating query simulators: an experiment using commercial searches and purchases. In: Agosti et al. [9], pp. 40–51 Huurnink, B., Hofmann, K., de Rijke, M., Bron, M.: Validating query simulators: an experiment using commercial searches and purchases. In: Agosti et al. [9], pp. 40–51
192.
go back to reference Imhof, M., Braschler, M.: Are test collections “real”? Mirroring real-world complexity in IR test collections. In: Mothe et al. [301], pp. 241–247 Imhof, M., Braschler, M.: Are test collections “real”? Mirroring real-world complexity in IR test collections. In: Mothe et al. [301], pp. 241–247
193.
go back to reference Inches, G., Crestani, F.: Overview of the international sexual predator identification competition at PAN-2012. In: Forner et al. [137] Inches, G., Crestani, F.: Overview of the international sexual predator identification competition at PAN-2012. In: Forner et al. [137]
194.
go back to reference Ionescu, B., et al.: ImageCLEF 2019: multimedia retrieval in medicine, lifelogging, security and nature. In: Crestani et al. [94] Ionescu, B., et al.: ImageCLEF 2019: multimedia retrieval in medicine, lifelogging, security and nature. In: Crestani et al. [94]
195.
go back to reference Ionescu, B., et al.: Overview of ImageCLEF 2017: information extraction from images. In: Jones et al. [209], pp. 315–337 Ionescu, B., et al.: Overview of ImageCLEF 2017: information extraction from images. In: Jones et al. [209], pp. 315–337
196.
go back to reference Ionescu, B., et al.: Overview of ImageCLEF 2018: challenges, datasets and evaluation. In: Bellot et al. [42], pp. 309–334 Ionescu, B., et al.: Overview of ImageCLEF 2018: challenges, datasets and evaluation. In: Bellot et al. [42], pp. 309–334
197.
go back to reference Jabeur, L.B., Soulier, L., Tamine, L., Mousset, P.: A product feature-based user-centric ranking model for e-commerce search. In: Fuhr et al. [143], pp. 174–186 Jabeur, L.B., Soulier, L., Tamine, L., Mousset, P.: A product feature-based user-centric ranking model for e-commerce search. In: Fuhr et al. [143], pp. 174–186
198.
go back to reference Jijkoun, V., de Rijke, M.: Overview of WebCLEF 2007. In: Peters et al. [366], pp. 725–731 Jijkoun, V., de Rijke, M.: Overview of WebCLEF 2007. In: Peters et al. [366], pp. 725–731
199.
go back to reference Jijkoun, V., de Rijke, M.: Overview of WebCLEF 2008. In: Peters et al. [359], pp. 787–793 Jijkoun, V., de Rijke, M.: Overview of WebCLEF 2008. In: Peters et al. [359], pp. 787–793
200.
go back to reference Joly, A., et al.: Overview of LifeCLEF 2018: a large-scale evaluation of species identification and recommendation algorithms in the era of AI. In: Bellot et al. [42], pp. 247–266 Joly, A., et al.: Overview of LifeCLEF 2018: a large-scale evaluation of species identification and recommendation algorithms in the era of AI. In: Bellot et al. [42], pp. 247–266
201.
go back to reference Joly, A., et al.: Overview of LifeCLEF 2019: identification of Amazonian plants, south & north American birds, and niche prediction. In: Crestani et al. [94] Joly, A., et al.: Overview of LifeCLEF 2019: identification of Amazonian plants, south & north American birds, and niche prediction. In: Crestani et al. [94]
202.
go back to reference Joly, A., et al.: LifeCLEF 2017 lab overview: multimedia species identification challenges. In: Jones et al. [209], pp. 255–274 Joly, A., et al.: LifeCLEF 2017 lab overview: multimedia species identification challenges. In: Jones et al. [209], pp. 255–274
203.
go back to reference Joly, A., et al.: Biodiversity information retrieval through large scale content-based identification: a long-term evaluation. In: Ferro and Peters [130] Joly, A., et al.: Biodiversity information retrieval through large scale content-based identification: a long-term evaluation. In: Ferro and Peters [130]
204.
go back to reference Joly, A., et al.: LifeCLEF 2014: multimedia life species identification challenges. In: Kanoulas et al. [216], pp. 229–249 Joly, A., et al.: LifeCLEF 2014: multimedia life species identification challenges. In: Kanoulas et al. [216], pp. 229–249
205.
go back to reference Joly, A., et al.: LifeCLEF 2015: multimedia life species identification challenges. In: Mothe et al. [301], pp. 462–483 Joly, A., et al.: LifeCLEF 2015: multimedia life species identification challenges. In: Mothe et al. [301], pp. 462–483
206.
go back to reference Joly, A., et al.: LifeCLEF 2016: multimedia life species identification challenges. In: Fuhr et al. [143], pp. 286–310 Joly, A., et al.: LifeCLEF 2016: multimedia life species identification challenges. In: Fuhr et al. [143], pp. 286–310
207.
go back to reference Jones, G.J.F.: ’Bout sound and vision: CLEF beyond text retrieval tasks. In: Ferro and Peters [130] Jones, G.J.F.: ’Bout sound and vision: CLEF beyond text retrieval tasks. In: Ferro and Peters [130]
208.
go back to reference Jones, G.J.F., Federico, M.: CLEF 2002 cross-language spoken document retrieval pilot track report. In: Peters et al. [355], pp. 446–457 Jones, G.J.F., Federico, M.: CLEF 2002 cross-language spoken document retrieval pilot track report. In: Peters et al. [355], pp. 446–457
210.
go back to reference Juola, P.: An overview of the traditional authorship attribution subtask. In: Forner et al. [137] Juola, P.: An overview of the traditional authorship attribution subtask. In: Forner et al. [137]
211.
go back to reference Jürgens, J., Hansen, P., Womser-Hacker, C.: Going beyond CLEF-IP: the ‘Reality’ for patent searchers. In: Catarci et al. [75], pp. 30–35 Jürgens, J., Hansen, P., Womser-Hacker, C.: Going beyond CLEF-IP: the ‘Reality’ for patent searchers. In: Catarci et al. [75], pp. 30–35
212.
go back to reference Kalpathy-Cramer, J., Müller, H., Bedrick, S., Eggel, I., Garcia Seco de Herrera, A., Tsikrika, T.: Overview of the CLEF 2011 medical image classification and retrieval tasks. In: Petras et al. [373] Kalpathy-Cramer, J., Müller, H., Bedrick, S., Eggel, I., Garcia Seco de Herrera, A., Tsikrika, T.: Overview of the CLEF 2011 medical image classification and retrieval tasks. In: Petras et al. [373]
213.
go back to reference Kamps, J., Koolen, M., Geva, S., Schenkel, R., SanJuan, E., Bogers, T.: From XML retrieval to semantic search and beyond. In: Ferro and Peters [130] Kamps, J., Koolen, M., Geva, S., Schenkel, R., SanJuan, E., Bogers, T.: From XML retrieval to semantic search and beyond. In: Ferro and Peters [130]
214.
go back to reference Kanoulas, E., Azzopardi, L.: CLEF 2017 dynamic search evaluation lab overview. In: Jones et al. [209], pp. 361–366 Kanoulas, E., Azzopardi, L.: CLEF 2017 dynamic search evaluation lab overview. In: Jones et al. [209], pp. 361–366
215.
go back to reference Kanoulas, E., Azzopardi, L., Hui Yang, G.: Overview of the CLEF dynamic search evaluation lab 2018. In: Bellot et al. [42], pp. 362–371 Kanoulas, E., Azzopardi, L., Hui Yang, G.: Overview of the CLEF dynamic search evaluation lab 2018. In: Bellot et al. [42], pp. 362–371
217.
go back to reference Karadzhov, G., Mihaylova, T., Kiprov, Y., Georgiev, G., Koychev, Y., Nakov, P.: The case for being average: a mediocrity approach to style masking and author obfuscation. In: Jones et al. [209], pp. 173–185 Karadzhov, G., Mihaylova, T., Kiprov, Y., Georgiev, G., Koychev, Y., Nakov, P.: The case for being average: a mediocrity approach to style masking and author obfuscation. In: Jones et al. [209], pp. 173–185
218.
go back to reference Karan, M., Snajder, J.: Evaluation of manual query expansion rules on a domain specific FAQ collection. In: Mothe et al. [301], pp. 248–253 Karan, M., Snajder, J.: Evaluation of manual query expansion rules on a domain specific FAQ collection. In: Mothe et al. [301], pp. 248–253
219.
go back to reference Karisani, P., Oroumchian, F., Rahgozar, M.: Tweet expansion method for filtering task in Twitter. In: Mothe et al. [301], pp. 55–64 Karisani, P., Oroumchian, F., Rahgozar, M.: Tweet expansion method for filtering task in Twitter. In: Mothe et al. [301], pp. 55–64
220.
go back to reference Karlgren, J.: Adopting systematic evaluation benchmarks in operational settings. In: Ferro and Peters [130] Karlgren, J.: Adopting systematic evaluation benchmarks in operational settings. In: Ferro and Peters [130]
221.
go back to reference Karlgren, J.: How lexical gold standards have effects on the usefulness of text analysis tools for digital scholarship. In: Crestani et al. [94] Karlgren, J.: How lexical gold standards have effects on the usefulness of text analysis tools for digital scholarship. In: Crestani et al. [94]
222.
go back to reference Karlgren, J., et al.: Evaluating learning language representations. In: Mothe et al. [301], pp. 254–260 Karlgren, J., et al.: Evaluating learning language representations. In: Mothe et al. [301], pp. 254–260
223.
go back to reference Karlgren, J., Gonzalo, J., Clough, P.: iCLEF 2006 overview: searching the Flickr WWW photo-sharing repository. In: Peters et al. [357], pp. 186–194 Karlgren, J., Gonzalo, J., Clough, P.: iCLEF 2006 overview: searching the Flickr WWW photo-sharing repository. In: Peters et al. [357], pp. 186–194
224.
go back to reference Karlgren, J., Hedlund, T., Järvelin, K., Keskustalo, H., Kettunen, K.: The challenges of language variation in information access. In: Ferro and Peters [130] Karlgren, J., Hedlund, T., Järvelin, K., Keskustalo, H., Kettunen, K.: The challenges of language variation in information access. In: Ferro and Peters [130]
225.
go back to reference Karlsson, V., Herman, P., Karlgren, J.: Evaluating categorisation in real life - an argument against simple but impractical metrics. In: Fuhr et al. [143], pp. 221–226 Karlsson, V., Herman, P., Karlgren, J.: Evaluating categorisation in real life - an argument against simple but impractical metrics. In: Fuhr et al. [143], pp. 221–226
226.
go back to reference Kavallieratou, E., del Blanco, C.R., Cuevas, C., García, N.: Interactive learning-based retrieval technique for visual lifelogging. In: Crestani et al. [94] Kavallieratou, E., del Blanco, C.R., Cuevas, C., García, N.: Interactive learning-based retrieval technique for visual lifelogging. In: Crestani et al. [94]
227.
go back to reference Kelly, L., Goeuriot, L., Suominen, H., Névéol, A., Palotti, J., Zuccon, G.: Overview of the CLEF eHealth evaluation lab 2016. In: Fuhr et al. [143], pp. 255–266 Kelly, L., Goeuriot, L., Suominen, H., Névéol, A., Palotti, J., Zuccon, G.: Overview of the CLEF eHealth evaluation lab 2016. In: Fuhr et al. [143], pp. 255–266
228.
go back to reference Kelly, L., et al.: Overview of the ShARe/CLEF eHealth evaluation lab 2014. In: Kanoulas et al. [216], pp. 172–191 Kelly, L., et al.: Overview of the ShARe/CLEF eHealth evaluation lab 2014. In: Kanoulas et al. [216], pp. 172–191
229.
go back to reference Keszler, A., Kovács, L., Szirányi, T.: The appearance of the giant component in descriptor graphs and its application for descriptor selection. In: Catarci et al. [75], pp. 76–81 Keszler, A., Kovács, L., Szirányi, T.: The appearance of the giant component in descriptor graphs and its application for descriptor selection. In: Catarci et al. [75], pp. 76–81
230.
go back to reference Kharazmi, S., Scholer, F., Vallet, D., Sanderson, M.: Examining additivity and weak baselines. ACM Trans. Inf. Syst. (TOIS) 34(4), 23:1–23:18 (2016)CrossRef Kharazmi, S., Scholer, F., Vallet, D., Sanderson, M.: Examining additivity and weak baselines. ACM Trans. Inf. Syst. (TOIS) 34(4), 23:1–23:18 (2016)CrossRef
231.
go back to reference Khwileh, A., Ganguly, D., Jones, G.J.F.: An investigation of cross-language information retrieval for user-generated internet video. In: Mothe et al. [301], pp. 117–129 Khwileh, A., Ganguly, D., Jones, G.J.F.: An investigation of cross-language information retrieval for user-generated internet video. In: Mothe et al. [301], pp. 117–129
232.
go back to reference Khwileh, A., Way, A., Jones, G.J.F.: Improving the reliability of query expansion for user-generated speech retrieval using query performance prediction. In: Jones et al. [209], pp. 43–56 Khwileh, A., Way, A., Jones, G.J.F.: Improving the reliability of query expansion for user-generated speech retrieval using query performance prediction. In: Jones et al. [209], pp. 43–56
233.
go back to reference Kille, B., et al.: Overview of NewsREEL’16: multi-dimensional evaluation of real-time stream-recommendation algorithms. In: Fuhr et al. [143], pp. 311–331 Kille, B., et al.: Overview of NewsREEL’16: multi-dimensional evaluation of real-time stream-recommendation algorithms. In: Fuhr et al. [143], pp. 311–331
234.
go back to reference Kille, B., et al.: Stream-based recommendations: online and offline evaluation as a service. In: Mothe et al. [301], pp. 497–517 Kille, B., et al.: Stream-based recommendations: online and offline evaluation as a service. In: Mothe et al. [301], pp. 497–517
235.
go back to reference Kim, S.J., Lee, J.H.: Subtopic mining based on head-modifier relation and co-occurrence of intents using web documents. In: Forner et al. [138], pp. 179–191 Kim, S.J., Lee, J.H.: Subtopic mining based on head-modifier relation and co-occurrence of intents using web documents. In: Forner et al. [138], pp. 179–191
236.
go back to reference Kliegr, T., Kuchar, J.: Benchmark of rule-based classifiers in the news recommendation task. In: Mothe et al. [301], pp. 130–141 Kliegr, T., Kuchar, J.: Benchmark of rule-based classifiers in the news recommendation task. In: Mothe et al. [301], pp. 130–141
237.
go back to reference Kluck, M.: The domain-specific track in CLEF 2004: overview of the results and remarks on the assessment process. In: Peters et al. [358], pp. 260–270 Kluck, M.: The domain-specific track in CLEF 2004: overview of the results and remarks on the assessment process. In: Peters et al. [358], pp. 260–270
238.
go back to reference Kluck, M., Gey, F.C.: The domain-specific task of CLEF - specific evaluation strategies in cross-language information retrieval. In: Peters [345], pp. 48–56CrossRef Kluck, M., Gey, F.C.: The domain-specific task of CLEF - specific evaluation strategies in cross-language information retrieval. In: Peters [345], pp. 48–56CrossRef
239.
go back to reference Kluck, M., Stempfhuber, M.: Domain-specific track CLEF 2005: overview of results and approaches, remarks on the assessment analysis. In: Peters et al. [365], pp. 212–221 Kluck, M., Stempfhuber, M.: Domain-specific track CLEF 2005: overview of results and approaches, remarks on the assessment analysis. In: Peters et al. [365], pp. 212–221
240.
go back to reference Kocher, M., Savoy, J.: Author clustering with an adaptive threshold. In: Jones et al. [209], pp. 186–198 Kocher, M., Savoy, J.: Author clustering with an adaptive threshold. In: Jones et al. [209], pp. 186–198
241.
go back to reference Koitka, S., Friedrich, C.M.: Optimized convolutional neural network ensembles for medical subfigure classification. In: Jones et al. [209], pp. 57–68 Koitka, S., Friedrich, C.M.: Optimized convolutional neural network ensembles for medical subfigure classification. In: Jones et al. [209], pp. 57–68
242.
go back to reference Koolen, M., et al.: Overview of the CLEF 2016 social book search lab. In: Fuhr et al. [143], pp. 351–370 Koolen, M., et al.: Overview of the CLEF 2016 social book search lab. In: Fuhr et al. [143], pp. 351–370
243.
go back to reference Koolen, M., et al.: Overview of the CLEF 2015 social book search lab. In: Mothe et al. [301], pp. 545–564 Koolen, M., et al.: Overview of the CLEF 2015 social book search lab. In: Mothe et al. [301], pp. 545–564
244.
go back to reference Koolen, M., Kazai, G., Kamps, J., Preminger, M., Doucet, A., Landoni, M.: Overview of the INEX 2012 social book search track. In: Forner et al. [137] Koolen, M., Kazai, G., Kamps, J., Preminger, M., Doucet, A., Landoni, M.: Overview of the INEX 2012 social book search track. In: Forner et al. [137]
245.
go back to reference Koops, H.V., Van Balen, J., Wiering, F.: Automatic segmentation and deep learning of bird sounds. In: Mothe et al. [301], pp. 261–267 Koops, H.V., Van Balen, J., Wiering, F.: Automatic segmentation and deep learning of bird sounds. In: Mothe et al. [301], pp. 261–267
246.
go back to reference Kordjamshidi, P., Rahgooy, T., Moens, M.F., Pustejovsky, J., Manzoor, U., Roberts, K.: CLEF 2017: multimodal spatial role labeling (mSpRL) task overview. In: Jones et al. [209], pp. 367–376 Kordjamshidi, P., Rahgooy, T., Moens, M.F., Pustejovsky, J., Manzoor, U., Roberts, K.: CLEF 2017: multimodal spatial role labeling (mSpRL) task overview. In: Jones et al. [209], pp. 367–376
247.
go back to reference Kosmopoulos, A., Paliouras, G., Androutsopoulos, I.: The effect of dimensionality reduction on large scale hierarchical classification. In: Kanoulas et al. [216], pp. 160–171 Kosmopoulos, A., Paliouras, G., Androutsopoulos, I.: The effect of dimensionality reduction on large scale hierarchical classification. In: Kanoulas et al. [216], pp. 160–171
248.
go back to reference Kumar, N.K., Santosh, G.S.K., Varma, V.: A language-independent approach to identify the named entities in under-resourced languages and clustering multilingual documents. In: Forner et al. [136], pp. 74–82 Kumar, N.K., Santosh, G.S.K., Varma, V.: A language-independent approach to identify the named entities in under-resourced languages and clustering multilingual documents. In: Forner et al. [136], pp. 74–82
249.
go back to reference Kurimo, M., Creutz, M., Varjokallio, M.: Morpho challenge evaluation using a linguistic gold standard. In: Peters et al. [366], pp. 864–872 Kurimo, M., Creutz, M., Varjokallio, M.: Morpho challenge evaluation using a linguistic gold standard. In: Peters et al. [366], pp. 864–872
250.
go back to reference Kurimo, M., Turunen, V.T., Varjokallio, M.: Overview of morpho challenge 2008. In: Peters et al. [359], pp. 951–966 Kurimo, M., Turunen, V.T., Varjokallio, M.: Overview of morpho challenge 2008. In: Peters et al. [359], pp. 951–966
251.
go back to reference Kurimo, M., Virpioja, S., Turunen, V.T., Blackwood, G.W., Byrne, W.: Overview and results of morpho challenge 2009. In: Peters et al. [360], pp. 587–597 Kurimo, M., Virpioja, S., Turunen, V.T., Blackwood, G.W., Byrne, W.: Overview and results of morpho challenge 2009. In: Peters et al. [360], pp. 587–597
252.
go back to reference Kürsten, J., Eibl, M.: Comparing IR system components using beanplots. In: Catarci et al. [75], pp. 136–137CrossRef Kürsten, J., Eibl, M.: Comparing IR system components using beanplots. In: Catarci et al. [75], pp. 136–137CrossRef
253.
go back to reference Kvist, M., Velupillai, S.: SCAN: a swedish clinical abbreviation normalizer - further development and adaptation to radiology. In: Kanoulas et al. [216], pp. 62–73 Kvist, M., Velupillai, S.: SCAN: a swedish clinical abbreviation normalizer - further development and adaptation to radiology. In: Kanoulas et al. [216], pp. 62–73
254.
go back to reference de Pertile, S.L., Pereira Moreira, V.: A test collection to evaluate plagiarism by missing or incorrect references. In: Catarci et al. [75], pp. 141–143 de Pertile, S.L., Pereira Moreira, V.: A test collection to evaluate plagiarism by missing or incorrect references. In: Catarci et al. [75], pp. 141–143
255.
go back to reference de Pertile, S.L., Rosso, P., Pereira Moreira, V.: Counting co-occurrences in citations to identify plagiarised text fragments. In: Forner et al. [138], pp. 150–154 de Pertile, S.L., Rosso, P., Pereira Moreira, V.: Counting co-occurrences in citations to identify plagiarised text fragments. In: Forner et al. [138], pp. 150–154
256.
go back to reference Lagopoulos, A., Anagnostou, A., Minas, A., Tsoumakas, G.: Learning-to-rank and relevance feedback for literature appraisal in empirical medicine. In: Bellot et al. [42], pp. 52–63 Lagopoulos, A., Anagnostou, A., Minas, A., Tsoumakas, G.: Learning-to-rank and relevance feedback for literature appraisal in empirical medicine. In: Bellot et al. [42], pp. 52–63
257.
go back to reference Lai, M., Tambuscio, M., Patti, V., Ruffo, G., Rosso, P.: Extracting graph topological information and users’ opinion. In: Jones et al. [209], pp. 112–118 Lai, M., Tambuscio, M., Patti, V., Ruffo, G., Rosso, P.: Extracting graph topological information and users’ opinion. In: Jones et al. [209], pp. 112–118
258.
go back to reference Landoni, M., Matteri, D., Murgia, E., Huibers, T., Soledad Pera, M.: Sonny, Cerca! evaluating the impact of using a vocal assistant to search at school. In: Crestani et al. [94] Landoni, M., Matteri, D., Murgia, E., Huibers, T., Soledad Pera, M.: Sonny, Cerca! evaluating the impact of using a vocal assistant to search at school. In: Crestani et al. [94]
259.
go back to reference Larsen, B.: The scholarly impact of CLEF 2010–2017. In: Ferro and Peters [130] Larsen, B.: The scholarly impact of CLEF 2010–2017. In: Ferro and Peters [130]
260.
go back to reference Larson, M., Newman, E., Jones, G.J.F.: Overview of VideoCLEF 2008: automatic generation of topic-based feeds for dual language audio-visual content. In: Peters et al. [359], pp. 906–917 Larson, M., Newman, E., Jones, G.J.F.: Overview of VideoCLEF 2008: automatic generation of topic-based feeds for dual language audio-visual content. In: Peters et al. [359], pp. 906–917
261.
go back to reference Larson, M., Newman, E., Jones, G.J.F.: Overview of VideoCLEF 2009: new perspectives on speech-based multimedia content enrichment. In: Peters et al. [368], pp. 354–368 Larson, M., Newman, E., Jones, G.J.F.: Overview of VideoCLEF 2009: new perspectives on speech-based multimedia content enrichment. In: Peters et al. [368], pp. 354–368
262.
go back to reference Lasseck, M.: Towards automatic large-scale identification of birds in audio recordings. In: Mothe et al. [301], pp. 364–375 Lasseck, M.: Towards automatic large-scale identification of birds in audio recordings. In: Mothe et al. [301], pp. 364–375
263.
go back to reference Leiva, L.A., Villegas, M., Paredes, R.: Relevant clouds: leveraging relevance feedback to build tag clouds for image search. In: Forner et al. [138], pp. 143–149 Leiva, L.A., Villegas, M., Paredes, R.: Relevant clouds: leveraging relevance feedback to build tag clouds for image search. In: Forner et al. [138], pp. 143–149
264.
go back to reference Leong, C.W., Hassan, S., Ruiz, M.E., Rada, M.: Improving query expansion for image retrieval via saliency and picturability. In: Forner et al. [136], pp. 137–142 Leong, C.W., Hassan, S., Ruiz, M.E., Rada, M.: Improving query expansion for image retrieval via saliency and picturability. In: Forner et al. [136], pp. 137–142
265.
go back to reference Lestari Paramita, M., Sanderson, M., Clough, P.: Diversity in photo retrieval: overview of the ImageCLEFPhoto task 2009. In: Peters et al. [368], pp. 45–59 Lestari Paramita, M., Sanderson, M., Clough, P.: Diversity in photo retrieval: overview of the ImageCLEFPhoto task 2009. In: Peters et al. [368], pp. 45–59
266.
go back to reference Li, P., Jiang, X., Kambhamettu, C., Shatkay, H.: Segmenting compound biomedical figures into their constituent panels. In: Jones et al. [209], pp. 199–210 Li, P., Jiang, X., Kambhamettu, C., Shatkay, H.: Segmenting compound biomedical figures into their constituent panels. In: Jones et al. [209], pp. 199–210
267.
go back to reference Li, W., Jones, G.J.F.: Enhancing medical information retrieval by exploiting a content-based recommender method. In: Mothe et al. [301], pp. 142–153 Li, W., Jones, G.J.F.: Enhancing medical information retrieval by exploiting a content-based recommender method. In: Mothe et al. [301], pp. 142–153
268.
go back to reference Linhares Pontes, E., Huet, S., Torres-Moreno, J.M.: Microblog contextualization: advantages and limitations of a multi-sentence compression approach. In: Bellot et al. [42], pp. 181–190 Linhares Pontes, E., Huet, S., Torres-Moreno, J.M.: Microblog contextualization: advantages and limitations of a multi-sentence compression approach. In: Bellot et al. [42], pp. 181–190
269.
go back to reference Lipani, A., Piroi, F., Andersson, L., Hanbury, A.: An information retrieval ontology for information retrieval nanopublications. In: Kanoulas et al. [216], pp. 44–49 Lipani, A., Piroi, F., Andersson, L., Hanbury, A.: An information retrieval ontology for information retrieval nanopublications. In: Kanoulas et al. [216], pp. 44–49
270.
go back to reference Litvinova, T., Seredin, P., Litvinova, O., Ryzhkova, E.: Estimating the similarities between texts of right-handed and left-handed males and females. In: Jones et al. [209], pp. 119–124 Litvinova, T., Seredin, P., Litvinova, O., Ryzhkova, E.: Estimating the similarities between texts of right-handed and left-handed males and females. In: Jones et al. [209], pp. 119–124
271.
go back to reference Liu, F., Peng, Y., Rosen, M.P.: An effective deep transfer learning and information fusion framework for medical visual question answering. In: Crestani et al. [94] Liu, F., Peng, Y., Rosen, M.P.: An effective deep transfer learning and information fusion framework for medical visual question answering. In: Crestani et al. [94]
272.
go back to reference Lommatzsch, A., et al.: CLEF 2017 NewsREEL overview: a stream-based recommender task for evaluation and education. In: Jones et al. [209], pp. 239–254 Lommatzsch, A., et al.: CLEF 2017 NewsREEL overview: a stream-based recommender task for evaluation and education. In: Jones et al. [209], pp. 239–254
273.
go back to reference Lommatzsch, A., Werner, S.: Optimizing and evaluating stream-based news recommendation algorithms. In: Mothe et al. [301], pp. 376–388 Lommatzsch, A., Werner, S.: Optimizing and evaluating stream-based news recommendation algorithms. In: Mothe et al. [301], pp. 376–388
274.
go back to reference Loponen, A., Järvelin, K.: A dictionary- and corpus-independent statistical lemmatizer for information retrieval in low resource languages. In: Agosti et al. [9], pp. 3–14 Loponen, A., Järvelin, K.: A dictionary- and corpus-independent statistical lemmatizer for information retrieval in low resource languages. In: Agosti et al. [9], pp. 3–14
275.
go back to reference Losada, D.E., Crestani, F.: A test collection for research on depression and language use. In: Fuhr et al. [143], pp. 28–39 Losada, D.E., Crestani, F.: A test collection for research on depression and language use. In: Fuhr et al. [143], pp. 28–39
276.
go back to reference Losada, D.E., Crestani, F., Parapar, J.: eRISK 2017: CLEF lab on early risk prediction on the internet: experimental foundations. In: Jones et al. [209], pp. 346–360 Losada, D.E., Crestani, F., Parapar, J.: eRISK 2017: CLEF lab on early risk prediction on the internet: experimental foundations. In: Jones et al. [209], pp. 346–360
277.
go back to reference Losada, D.E., Crestani, F., Parapar, J.: Overview of eRisk: early risk prediction on the internet. In: Bellot et al. [42], pp. 343–361 Losada, D.E., Crestani, F., Parapar, J.: Overview of eRisk: early risk prediction on the internet. In: Bellot et al. [42], pp. 343–361
278.
go back to reference Losada, D.E., Crestani, F., Parapar, J.: Overview of eRisk 2019: early risk prediction on the internet. In: Crestani et al. [94] Losada, D.E., Crestani, F., Parapar, J.: Overview of eRisk 2019: early risk prediction on the internet. In: Crestani et al. [94]
279.
go back to reference Mackie, S., McCreadie, R., Macdonald, C., Ounis, I.: Comparing algorithms for microblog summarisation. In: Kanoulas et al. [216], pp. 153–159 Mackie, S., McCreadie, R., Macdonald, C., Ounis, I.: Comparing algorithms for microblog summarisation. In: Kanoulas et al. [216], pp. 153–159
280.
go back to reference Magdy, W., Jones, G.J.F.: Examining the robustness of evaluation metrics for patent retrieval with incomplete relevance judgements. In: Agosti et al. [9], pp. 82–93 Magdy, W., Jones, G.J.F.: Examining the robustness of evaluation metrics for patent retrieval with incomplete relevance judgements. In: Agosti et al. [9], pp. 82–93
281.
go back to reference Magnini, B., et al.: Overview of the CLEF 2006 multilingual question answering track. In: Peters et al. [357], pp. 223–256 Magnini, B., et al.: Overview of the CLEF 2006 multilingual question answering track. In: Peters et al. [357], pp. 223–256
282.
go back to reference Magnini, B., et al.: The multiple language question answering track at CLEF 2003. In: Peters et al. [356], pp. 471–486 Magnini, B., et al.: The multiple language question answering track at CLEF 2003. In: Peters et al. [356], pp. 471–486
283.
go back to reference Magnini, B., et al.: Overview of the CLEF 2004 multilingual question answering track. In: Peters et al. [358], pp. 371–391 Magnini, B., et al.: Overview of the CLEF 2004 multilingual question answering track. In: Peters et al. [358], pp. 371–391
284.
go back to reference Mandl, T., et al.: LogCLEF 2009: the CLEF 2009 multilingual logfile analysis track overview. In: Peters et al. [360], pp. 508–517 Mandl, T., et al.: LogCLEF 2009: the CLEF 2009 multilingual logfile analysis track overview. In: Peters et al. [360], pp. 508–517
285.
go back to reference Mandl, T., et al.: GeoCLEF 2008: the CLEF 2008 cross-language geographic information retrieval track overview. In: Peters et al. [359], pp. 808–821 Mandl, T., et al.: GeoCLEF 2008: the CLEF 2008 cross-language geographic information retrieval track overview. In: Peters et al. [359], pp. 808–821
286.
go back to reference Mandl, T., Di Nunzio, G.M., Schulz, J.M.: LogCLEF 2010: the CLEF 2010 multilingual logfile analysis track overview. In: Braschler et al. [61] Mandl, T., Di Nunzio, G.M., Schulz, J.M.: LogCLEF 2010: the CLEF 2010 multilingual logfile analysis track overview. In: Braschler et al. [61]
287.
go back to reference Mandl, T., et al.: GeoCLEF 2007: the CLEF 2007 cross-language geographic information retrieval track overview. In: Peters et al. [366], pp. 745–772 Mandl, T., et al.: GeoCLEF 2007: the CLEF 2007 cross-language geographic information retrieval track overview. In: Peters et al. [366], pp. 745–772
288.
go back to reference Manotumruksa, J., Macdonald, C., Ounis, I.: Predicting contextually appropriate venues in location-based social networks. In: Fuhr et al. [143], pp. 96–109 Manotumruksa, J., Macdonald, C., Ounis, I.: Predicting contextually appropriate venues in location-based social networks. In: Fuhr et al. [143], pp. 96–109
289.
go back to reference Martínez-Gómez, J., García-Varea, I., Caputo, B.: Overview of the ImageCLEF 2012 robot vision task. In: Forner et al. [137] Martínez-Gómez, J., García-Varea, I., Caputo, B.: Overview of the ImageCLEF 2012 robot vision task. In: Forner et al. [137]
290.
go back to reference Mayfield, J., Lawrie, D., McNamee, P., Oard, D.W.: Building a cross-language entity linking collection in twenty-one languages. In: Forner et al. [136], pp. 3–13 Mayfield, J., Lawrie, D., McNamee, P., Oard, D.W.: Building a cross-language entity linking collection in twenty-one languages. In: Forner et al. [136], pp. 3–13
291.
go back to reference McCreadie, R., Macdonald, C., Ounis, I., Brassey, J.: A study of personalised medical literature search. In: Kanoulas et al. [216], pp. 74–85 McCreadie, R., Macdonald, C., Ounis, I., Brassey, J.: A study of personalised medical literature search. In: Kanoulas et al. [216], pp. 74–85
292.
go back to reference McMinn, A.J., Jose, J.M.: Real-time entity-based event detection for Twitter. In: Mothe et al. [301], pp. 65–77 McMinn, A.J., Jose, J.M.: Real-time entity-based event detection for Twitter. In: Mothe et al. [301], pp. 65–77
293.
go back to reference Medvedeva, M., Haagsma, H., Nissim, M.: An analysis of cross-genre and in-genre performance for author profiling in social media. In: Jones et al. [209], pp. 211–223 Medvedeva, M., Haagsma, H., Nissim, M.: An analysis of cross-genre and in-genre performance for author profiling in social media. In: Jones et al. [209], pp. 211–223
294.
go back to reference Miftahutdinov, Z., Tutubalina, E.: Deep learning for ICD coding: looking for medical concepts in clinical documents in English and in French. In: Bellot et al. [42], pp. 203–215 Miftahutdinov, Z., Tutubalina, E.: Deep learning for ICD coding: looking for medical concepts in clinical documents in English and in French. In: Bellot et al. [42], pp. 203–215
295.
go back to reference Mirsarraf, M.R., Dehghani, N.: A dependency-inspired semantic evaluation of machine translation systems. In: Forner et al. [138], pp. 71–74 Mirsarraf, M.R., Dehghani, N.: A dependency-inspired semantic evaluation of machine translation systems. In: Forner et al. [138], pp. 71–74
296.
go back to reference Mitrovic, S., Müller, H.: Summarizing citation contexts of scientific publications. In: Mothe et al. [301], pp. 154–165 Mitrovic, S., Müller, H.: Summarizing citation contexts of scientific publications. In: Mothe et al. [301], pp. 154–165
297.
go back to reference Molina, A., SanJuan, E., Torres-Moreno, J.M.: A Turing test to evaluate a complex summarization task. In: Forner et al. [138], pp. 75–80 Molina, A., SanJuan, E., Torres-Moreno, J.M.: A Turing test to evaluate a complex summarization task. In: Forner et al. [138], pp. 75–80
298.
go back to reference Molina, S., Mothe, J., Roques, D., Tanguy, L., Ullah, M.Z.: IRIT-QFR: IRIT query feature resource. In: Jones et al. [209], pp. 69–81 Molina, S., Mothe, J., Roques, D., Tanguy, L., Ullah, M.Z.: IRIT-QFR: IRIT query feature resource. In: Jones et al. [209], pp. 69–81
299.
go back to reference Morante, R., Daelemans, W.: Overview of the QA4MRE pilot task: annotating modality and negation for a machine reading evaluation. In: Petras et al. [373] Morante, R., Daelemans, W.: Overview of the QA4MRE pilot task: annotating modality and negation for a machine reading evaluation. In: Petras et al. [373]
300.
go back to reference Moreno, R., Huáng, W., Younus, A., O’Mahony, M.P., Hurley, N.J.: Evaluation of hierarchical clustering via markov decision processes for efficient navigation and search. In: Jones et al. [209], pp. 125–131 Moreno, R., Huáng, W., Younus, A., O’Mahony, M.P., Hurley, N.J.: Evaluation of hierarchical clustering via markov decision processes for efficient navigation and search. In: Jones et al. [209], pp. 125–131
302.
go back to reference Mulhem, P., Goeuriot, L., Dogra, N., Amer, N.O.: TimeLine illustration based on microblogs: when diversification meets metadata re-ranking. In: Jones et al. [209], pp. 224–235 Mulhem, P., Goeuriot, L., Dogra, N., Amer, N.O.: TimeLine illustration based on microblogs: when diversification meets metadata re-ranking. In: Jones et al. [209], pp. 224–235
304.
go back to reference Müller, H., Deselaers, T., Deserno, T.M., Clough, P., Kim, E., Hersh, W.R.: Overview of the ImageCLEFmed 2006 medical retrieval and medical annotation tasks. In: Peters et al. [357], pp. 595–608 Müller, H., Deselaers, T., Deserno, T.M., Clough, P., Kim, E., Hersh, W.R.: Overview of the ImageCLEFmed 2006 medical retrieval and medical annotation tasks. In: Peters et al. [357], pp. 595–608
305.
go back to reference Müller, H., Deselaers, T., Deserno, T.M., Kalpathy-Cramer, J., Kim, E., Hersh, W.R.: Overview of the ImageCLEFmed 2007 medical retrieval and medical annotation tasks. In: Peters et al. [366], pp. 472–491 Müller, H., Deselaers, T., Deserno, T.M., Kalpathy-Cramer, J., Kim, E., Hersh, W.R.: Overview of the ImageCLEFmed 2007 medical retrieval and medical annotation tasks. In: Peters et al. [366], pp. 472–491
306.
go back to reference Müller, H., Hanbury, A.: EaaS: evaluation-as-a-service and experiences from the VISCERAL project. In: Ferro and Peters [130] Müller, H., Hanbury, A.: EaaS: evaluation-as-a-service and experiences from the VISCERAL project. In: Ferro and Peters [130]
307.
go back to reference Müller, H., Garcia Seco de Herrera, A., Kalpathy-Cramer, J., Demner-Fushman, D., Antani, S., Eggel, I.: Overview of the ImageCLEF 2012 medical image retrieval and classification tasks. In: Forner et al. [137] Müller, H., Garcia Seco de Herrera, A., Kalpathy-Cramer, J., Demner-Fushman, D., Antani, S., Eggel, I.: Overview of the ImageCLEF 2012 medical image retrieval and classification tasks. In: Forner et al. [137]
308.
go back to reference Müller, H., et al.: Overview of the CLEF 2009 medical image retrieval track. In: Peters et al. [368], pp. 72–84 Müller, H., et al.: Overview of the CLEF 2009 medical image retrieval track. In: Peters et al. [368], pp. 72–84
309.
go back to reference Müller, H., et al.: Overview of the CLEF 2010 medical image retrieval track. In: Braschler et al. [61] Müller, H., et al.: Overview of the CLEF 2010 medical image retrieval track. In: Braschler et al. [61]
310.
go back to reference Müller, H., Kalpathy-Cramer, J., Garcia Seco de Herrera, A.: Experiences from the ImageCLEF medical retrieval and annotation tasks. In: Ferro and Peters [130] Müller, H., Kalpathy-Cramer, J., Garcia Seco de Herrera, A.: Experiences from the ImageCLEF medical retrieval and annotation tasks. In: Ferro and Peters [130]
311.
go back to reference Müller, H., Kalpathy-Cramer, J., Kahn, C.E., Hatt, W., Bedrick, S., Hersh, W.: Overview of the ImageCLEFmed 2008 medical image retrieval task. In: Peters et al. [359], pp. 512–522 Müller, H., Kalpathy-Cramer, J., Kahn, C.E., Hatt, W., Bedrick, S., Hersh, W.: Overview of the ImageCLEFmed 2008 medical image retrieval task. In: Peters et al. [359], pp. 512–522
312.
go back to reference Murauer, B., Specht, G.: Generating cross-domain text corpora from social media comments. In: Crestani et al. [94] Murauer, B., Specht, G.: Generating cross-domain text corpora from social media comments. In: Crestani et al. [94]
313.
go back to reference Nakov, P., et al.: Overview of the CLEF-2018 CheckThat! Lab on automatic identification and verification of political claims. In: Bellot et al. [42], pp. 372–387 Nakov, P., et al.: Overview of the CLEF-2018 CheckThat! Lab on automatic identification and verification of political claims. In: Bellot et al. [42], pp. 372–387
316.
go back to reference Nordlie, R., Pharo, N.: Seven years of INEX interactive retrieval experiments - lessons and challenges. In: Catarci et al. [75], pp. 13–23 Nordlie, R., Pharo, N.: Seven years of INEX interactive retrieval experiments - lessons and challenges. In: Catarci et al. [75], pp. 13–23
317.
go back to reference Nowak, S., Dunker, P.: Overview of the CLEF 2009 large-scale visual concept detection and annotation task. In: Peters et al. [368], pp. 94–109 Nowak, S., Dunker, P.: Overview of the CLEF 2009 large-scale visual concept detection and annotation task. In: Peters et al. [368], pp. 94–109
318.
go back to reference Nowak, S., Huiskes, M.J.: New strategies for image annotation: overview of the photo annotation task at ImageCLEF 2010. In: Braschler et al. [61] Nowak, S., Huiskes, M.J.: New strategies for image annotation: overview of the photo annotation task at ImageCLEF 2010. In: Braschler et al. [61]
319.
go back to reference Nowak, S., Nagel, K., Liebetrau, J.: The CLEF 2011 photo annotation and concept-based retrieval tasks. In: Petras et al. [373] Nowak, S., Nagel, K., Liebetrau, J.: The CLEF 2011 photo annotation and concept-based retrieval tasks. In: Petras et al. [373]
320.
go back to reference Oard, D.W., Gonzalo, J.: The CLEF 2001 interactive track. In: Peters et al. [354], pp. 308–319 Oard, D.W., Gonzalo, J.: The CLEF 2001 interactive track. In: Peters et al. [354], pp. 308–319
321.
go back to reference Oard, D.W., Gonzalo, J.: The CLEF 2003 interactive track. In: Peters et al. [356], pp. 425–434 Oard, D.W., Gonzalo, J.: The CLEF 2003 interactive track. In: Peters et al. [356], pp. 425–434
322.
go back to reference Oard, D.W., et al.: Overview of the CLEF-2006 cross-language speech retrieval track. In: Peters et al. [357], pp. 744–758 Oard, D.W., et al.: Overview of the CLEF-2006 cross-language speech retrieval track. In: Peters et al. [357], pp. 744–758
323.
go back to reference Oh, H.S., Jung, Y., Kim, K.Y.: A multiple-stage approach to re-ranking medical documents. In: Mothe et al. [301], pp. 166–177 Oh, H.S., Jung, Y., Kim, K.Y.: A multiple-stage approach to re-ranking medical documents. In: Mothe et al. [301], pp. 166–177
324.
go back to reference Olvera-Lobo, M.D., Gutiérrez-Artacho, J.: Multilingual question-answering system in biomedical domain on the web: an evaluation. In: Forner et al. [136], pp. 83–88 Olvera-Lobo, M.D., Gutiérrez-Artacho, J.: Multilingual question-answering system in biomedical domain on the web: an evaluation. In: Forner et al. [136], pp. 83–88
325.
go back to reference Orio, N., Liem, C.C.S., Peeters, G., Schedl, M.: MusiClef: multimodal music tagging task. In: Catarci et al. [75], pp. 36–41 Orio, N., Liem, C.C.S., Peeters, G., Schedl, M.: MusiClef: multimodal music tagging task. In: Catarci et al. [75], pp. 36–41
326.
go back to reference Orio, N., Rizo, D.: Overview of MusiCLEF 2011. In: Petras et al. [373] Orio, N., Rizo, D.: Overview of MusiCLEF 2011. In: Petras et al. [373]
327.
go back to reference Ortega-Mendoza, M., Franco-Arcega, A., López-Monroy, A.P., Montes-y Gómez, M.: I, me, mine: the role of personal phrases in author profiling. In: Fuhr et al. [143], pp. 110–122 Ortega-Mendoza, M., Franco-Arcega, A., López-Monroy, A.P., Montes-y Gómez, M.: I, me, mine: the role of personal phrases in author profiling. In: Fuhr et al. [143], pp. 110–122
328.
go back to reference Otterbacher, J.: Addressing social bias in information retrieval. In: Bellot et al. [42], pp. 121–127 Otterbacher, J.: Addressing social bias in information retrieval. In: Bellot et al. [42], pp. 121–127
329.
go back to reference Pääkkönen, T., et al.: Exploring behavioral dimensions in session effectiveness. In: Mothe et al. [301], pp. 178–189 Pääkkönen, T., et al.: Exploring behavioral dimensions in session effectiveness. In: Mothe et al. [301], pp. 178–189
330.
go back to reference Palotti, J., Zuccon, G., Bernhardt, J., Hanbury, A., Goeuriot, L.: Assessors agreement: a case study across assessor type, payment levels, query variations and relevance dimensions. In: Fuhr et al. [143], pp. 40–53 Palotti, J., Zuccon, G., Bernhardt, J., Hanbury, A., Goeuriot, L.: Assessors agreement: a case study across assessor type, payment levels, query variations and relevance dimensions. In: Fuhr et al. [143], pp. 40–53
331.
go back to reference Parks, M., Karlgren, J., Stymne, S.: Plausibility testing for lexical resources. In: Jones et al. [209], pp. 132–137 Parks, M., Karlgren, J., Stymne, S.: Plausibility testing for lexical resources. In: Jones et al. [209], pp. 132–137
332.
go back to reference Pasi, G., et al.: Evaluation of personalised information retrieval at CLEF 2018 (PIR-CLEF). In: Bellot et al. [42], pp. 335–342 Pasi, G., et al.: Evaluation of personalised information retrieval at CLEF 2018 (PIR-CLEF). In: Bellot et al. [42], pp. 335–342
333.
go back to reference Pasi, G., Jones, G.J.F., Goeuriot, L., Kelly, L., Marrara, S., Sanvitto, C.: Overview of the CLEF 2019 personalised information retrieval lab (PIR-CLEF 2019). In: Crestani et al. [94] Pasi, G., Jones, G.J.F., Goeuriot, L., Kelly, L., Marrara, S., Sanvitto, C.: Overview of the CLEF 2019 personalised information retrieval lab (PIR-CLEF 2019). In: Crestani et al. [94]
334.
go back to reference Pasi, G., Jones, G.J.F., Marrara, S., Sanvitto, C., Ganguly, D., Sen, P.: Overview of the CLEF 2017 personalised information retrieval pilot lab (PIR-CLEF 2017). In: Jones et al. [209], pp. 338–345 Pasi, G., Jones, G.J.F., Marrara, S., Sanvitto, C., Ganguly, D., Sen, P.: Overview of the CLEF 2017 personalised information retrieval pilot lab (PIR-CLEF 2017). In: Jones et al. [209], pp. 338–345
335.
go back to reference Pecina, P., Hoffmannová, P., Jones, G.J.F., Zhang, Y., Oard, D.W.: Overview of the CLEF-2007 cross-language speech retrieval track. In: Peters et al. [366], pp. 674–686 Pecina, P., Hoffmannová, P., Jones, G.J.F., Zhang, Y., Oard, D.W.: Overview of the CLEF-2007 cross-language speech retrieval track. In: Peters et al. [366], pp. 674–686
336.
go back to reference Pellegrin, L., et al.: A two-step retrieval method for image captioning. In: Fuhr et al. [143], pp. 150–161 Pellegrin, L., et al.: A two-step retrieval method for image captioning. In: Fuhr et al. [143], pp. 150–161
337.
go back to reference Peñas, A., Forner, P., Rodrigo, A., Sutcliffe, R.F.E., Forascu, C., Mota, C.: Overview of ResPubliQA 2010: question answering evaluation over European legislation. In: Braschler et al. [61] Peñas, A., Forner, P., Rodrigo, A., Sutcliffe, R.F.E., Forascu, C., Mota, C.: Overview of ResPubliQA 2010: question answering evaluation over European legislation. In: Braschler et al. [61]
338.
go back to reference Peñas, A., et al.: Overview of ResPubliQA 2009: question answering evaluation over european legislation. In: Peters et al. [360], pp. 174–196 Peñas, A., et al.: Overview of ResPubliQA 2009: question answering evaluation over european legislation. In: Peters et al. [360], pp. 174–196
339.
go back to reference Peñas, A., et al.: Overview of QA4MRE at CLEF 2011: question answering for machine reading evaluation. In: Petras et al. [373] Peñas, A., et al.: Overview of QA4MRE at CLEF 2011: question answering for machine reading evaluation. In: Petras et al. [373]
340.
go back to reference Peñas, A., Hovy, E.H., Forner, P., Rodrigo, A., Sutcliffe, R.F.E., Morante, R.: QA4MRE 2011–2013: overview of question answering for machine reading evaluation. In: Forner et al. [138], pp. 303–320 Peñas, A., Hovy, E.H., Forner, P., Rodrigo, A., Sutcliffe, R.F.E., Morante, R.: QA4MRE 2011–2013: overview of question answering for machine reading evaluation. In: Forner et al. [138], pp. 303–320
341.
go back to reference Peñas, A., et al.: Overview of QA4MRE at CLEF 2012: question answering for machine reading evaluation. In: Forner et al. [137] Peñas, A., et al.: Overview of QA4MRE at CLEF 2012: question answering for machine reading evaluation. In: Forner et al. [137]
342.
go back to reference Peñas, A., et al.: Results and lessons of the question answering track at CLEF. In: Ferro and Peters [130] Peñas, A., et al.: Results and lessons of the question answering track at CLEF. In: Ferro and Peters [130]
343.
go back to reference Peñas, A., Unger, C., Ngonga Ngomo, A.C.a.: Overview of CLEF question answering track 2014. In: Kanoulas et al. [216], pp. 300–306 Peñas, A., Unger, C., Ngonga Ngomo, A.C.a.: Overview of CLEF question answering track 2014. In: Kanoulas et al. [216], pp. 300–306
344.
go back to reference Peñas, A., Unger, C., Paliouras, P., Kakadiaris, I.A.: Overview of the CLEF question answering track 2015. In: Mothe et al. [301], pp. 539–544 Peñas, A., Unger, C., Paliouras, P., Kakadiaris, I.A.: Overview of the CLEF question answering track 2015. In: Mothe et al. [301], pp. 539–544
346.
go back to reference Peters, C.: Introduction. In: Cross-Language Information Retrieval and Evaluation: Workshop of Cross-Language Evaluation Forum (CLEF 2000) [345], pp. 1–6 Peters, C.: Introduction. In: Cross-Language Information Retrieval and Evaluation: Workshop of Cross-Language Evaluation Forum (CLEF 2000) [345], pp. 1–6
347.
go back to reference Peters, C.: What happened in CLEF 2004? In: Peters et al. [358], pp. 1–9 Peters, C.: What happened in CLEF 2004? In: Peters et al. [358], pp. 1–9
348.
go back to reference Peters, C.: What happened in CLEF 2005? In: Peters et al. [365], pp. 1–10 Peters, C.: What happened in CLEF 2005? In: Peters et al. [365], pp. 1–10
349.
go back to reference Peters, C.: What happened in CLEF 2006? In: Peters et al. [357], pp. 1–10CrossRef Peters, C.: What happened in CLEF 2006? In: Peters et al. [357], pp. 1–10CrossRef
350.
go back to reference Peters, C.: What happened in CLEF 2007? In: Peters et al. [366], pp. 1–12 Peters, C.: What happened in CLEF 2007? In: Peters et al. [366], pp. 1–12
351.
go back to reference Peters, C.: What happened in CLEF 2008? In: Peters et al. [359], pp. 1–14 Peters, C.: What happened in CLEF 2008? In: Peters et al. [359], pp. 1–14
352.
go back to reference Peters, C.: What happened in CLEF 2009? In: Peters et al. [360], pp. 1–12 Peters, C.: What happened in CLEF 2009? In: Peters et al. [360], pp. 1–12
369.
go back to reference Petras, V., Baerisch, S.: The domain-specific track at CLEF 2008. In: Peters et al. [359], pp. 186–198 Petras, V., Baerisch, S.: The domain-specific track at CLEF 2008. In: Peters et al. [359], pp. 186–198
370.
go back to reference Petras, V., Baerisch, S., Stempfhuber, M.: The domain-specific track at CLEF 2007. In: Peters et al. [366], pp. 160–173 Petras, V., Baerisch, S., Stempfhuber, M.: The domain-specific track at CLEF 2007. In: Peters et al. [366], pp. 160–173
371.
go back to reference Petras, V., et al.: Cultural heritage in CLEF (CHiC) 2013. In: Forner et al. [138], pp. 192–211 Petras, V., et al.: Cultural heritage in CLEF (CHiC) 2013. In: Forner et al. [138], pp. 192–211
372.
go back to reference Petras, V., et al.: Cultural heritage in CLEF (CHiC) overview 2012. In: Forner et al. [137] Petras, V., et al.: Cultural heritage in CLEF (CHiC) overview 2012. In: Forner et al. [137]
374.
go back to reference Piras, L., Caputo, B., Dang-Nguyen, D.T., Riegler, M., Halvorsen, P.: Image retrieval evaluation in specific domains. In: Ferro and Peters [130] Piras, L., Caputo, B., Dang-Nguyen, D.T., Riegler, M., Halvorsen, P.: Image retrieval evaluation in specific domains. In: Ferro and Peters [130]
375.
go back to reference Piroi, F.: CLEF-IP 2010: retrieval experiments in the intellectual property domain. In: Braschler et al. [61] Piroi, F.: CLEF-IP 2010: retrieval experiments in the intellectual property domain. In: Braschler et al. [61]
376.
go back to reference Piroi, F., Hanbury, A.: Multilingual patent text retrieval evaluation: CLEF-IP. In: Ferro and Peters [130] Piroi, F., Hanbury, A.: Multilingual patent text retrieval evaluation: CLEF-IP. In: Ferro and Peters [130]
377.
go back to reference Piroi, F., Lupu, M., Hanbury, A.: Effects of language and topic size in patent IR: an empirical study. In: Catarci et al. [75], pp. 54–66 Piroi, F., Lupu, M., Hanbury, A.: Effects of language and topic size in patent IR: an empirical study. In: Catarci et al. [75], pp. 54–66
378.
go back to reference Piroi, F., Lupu, M., Hanbury, A.: Overview of CLEF-IP 2013 lab - information retrieval in the patent domain. In: Forner et al. [138], pp. 232–249 Piroi, F., Lupu, M., Hanbury, A.: Overview of CLEF-IP 2013 lab - information retrieval in the patent domain. In: Forner et al. [138], pp. 232–249
379.
go back to reference Piroi, F., Lupu, M., Hanbury, A., Sexton, A.P., Magdy, W., Filippov, I.V.: CLEF-IP 2012: retrieval experiments in the intellectual property domain. In: Forner et al. [137] Piroi, F., Lupu, M., Hanbury, A., Sexton, A.P., Magdy, W., Filippov, I.V.: CLEF-IP 2012: retrieval experiments in the intellectual property domain. In: Forner et al. [137]
380.
go back to reference Piroi, F., Lupu, M., Hanbury, A., Zenz, V.: CLEF-IP 2011: retrieval in the intellectual property domain. In: Petras et al. [373] Piroi, F., Lupu, M., Hanbury, A., Zenz, V.: CLEF-IP 2011: retrieval in the intellectual property domain. In: Petras et al. [373]
381.
go back to reference Popescu, A., Tsikrika, T., Kludas, J.: Overview of the Wikipedia retrieval task at ImageCLEF 2010. In: Braschler et al. [61] Popescu, A., Tsikrika, T., Kludas, J.: Overview of the Wikipedia retrieval task at ImageCLEF 2010. In: Braschler et al. [61]
382.
go back to reference Potha, N., Stamatatos, E.: An improved impostors method for authorship verification. In: Jones et al. [209], pp. 138–144 Potha, N., Stamatatos, E.: An improved impostors method for authorship verification. In: Jones et al. [209], pp. 138–144
383.
go back to reference Potthast, M., Barrón-Cedeño, A., Eiselt, A., Stein, B., Rosso, P.: Overview of the 2nd international competition on plagiarism detection. In: Braschler et al. [61] Potthast, M., Barrón-Cedeño, A., Eiselt, A., Stein, B., Rosso, P.: Overview of the 2nd international competition on plagiarism detection. In: Braschler et al. [61]
384.
go back to reference Potthast, M., Eiselt, A., Barrón-Cedeño, A., Stein, B., Rosso, P.: Overview of the 3rd international competition on plagiarism detection. In: Petras et al. [373] Potthast, M., Eiselt, A., Barrón-Cedeño, A., Stein, B., Rosso, P.: Overview of the 3rd international competition on plagiarism detection. In: Petras et al. [373]
385.
go back to reference Potthast, M., et al.: Overview of the 4th international competition on plagiarism detection. In: Forner et al. [137] Potthast, M., et al.: Overview of the 4th international competition on plagiarism detection. In: Forner et al. [137]
386.
go back to reference Potthast, M., Gollub, T., Rangel Pardo, F., Rosso, P., Stamatatos, E., Stein, B.: Improving the reproducibility of PAN’s shared tasks: plagiarism detection, author identification, and author profiling. In: Kanoulas et al. [216], pp. 268–299 Potthast, M., Gollub, T., Rangel Pardo, F., Rosso, P., Stamatatos, E., Stein, B.: Improving the reproducibility of PAN’s shared tasks: plagiarism detection, author identification, and author profiling. In: Kanoulas et al. [216], pp. 268–299
387.
go back to reference Potthast, M., Gollub, T., Wiegmann, M., Stein, b.: TIRA integrated research architecture. In: Ferro and Peters [130] Potthast, M., Gollub, T., Wiegmann, M., Stein, b.: TIRA integrated research architecture. In: Ferro and Peters [130]
388.
go back to reference Potthast, M., Holfeld, T.: Overview of the 2nd international competition on Wikipedia vandalism detection. In: Petras et al. [373] Potthast, M., Holfeld, T.: Overview of the 2nd international competition on Wikipedia vandalism detection. In: Petras et al. [373]
389.
go back to reference Potthast, M., Rangel Pardo, F.M., Tschuggnall, M., Stamatatos, E., Rosso, P., Stein, B.: Overview of PAN’17 - author identification, author profiling, and author obfuscation. In: Jones et al. [209], pp. 275–290 Potthast, M., Rangel Pardo, F.M., Tschuggnall, M., Stamatatos, E., Rosso, P., Stein, B.: Overview of PAN’17 - author identification, author profiling, and author obfuscation. In: Jones et al. [209], pp. 275–290
390.
go back to reference Potthast, M., Stein, B., Holfeld, T.: Overview of the 1st international competition on Wikipedia vandalism detection. In: Braschler et al. [61] Potthast, M., Stein, B., Holfeld, T.: Overview of the 1st international competition on Wikipedia vandalism detection. In: Braschler et al. [61]
391.
go back to reference Pritsos, D.A., Stamatatos, E.: The impact of noise in web genre identification. In: Mothe et al. [301], pp. 268–273 Pritsos, D.A., Stamatatos, E.: The impact of noise in web genre identification. In: Mothe et al. [301], pp. 268–273
392.
go back to reference Pronobis, A., Fornoni, M., Christensen, H.I., Caputo, B.: The Robot Vision Track at ImageCLEF 2010. In: Braschler et al. [61] Pronobis, A., Fornoni, M., Christensen, H.I., Caputo, B.: The Robot Vision Track at ImageCLEF 2010. In: Braschler et al. [61]
393.
go back to reference Pronobis, A., Xing, L., Caputo, B.: Overview of the CLEF 2009 robot vision track. In: Peters et al. [368], pp. 110–119 Pronobis, A., Xing, L., Caputo, B.: Overview of the CLEF 2009 robot vision track. In: Peters et al. [368], pp. 110–119
394.
go back to reference Raghavi, K.C., Chinnakotla, M.K., Black, A.W., Shrivastava, M.: WebShodh: a code mixed factoid question answering system for web. In: Jones et al. [209], pp. 104–111 Raghavi, K.C., Chinnakotla, M.K., Black, A.W., Shrivastava, M.: WebShodh: a code mixed factoid question answering system for web. In: Jones et al. [209], pp. 104–111
395.
go back to reference Ragheb, W., Azé, J., Bringay, S., Servajean, M.: Language modeling in temporal mood variation models for early risk detection on the internet. In: Crestani et al. [94] Ragheb, W., Azé, J., Bringay, S., Servajean, M.: Language modeling in temporal mood variation models for early risk detection on the internet. In: Crestani et al. [94]
396.
go back to reference Rangel Pardo, F., Rosso, P.: On the multilingual and genre robustness of EmoGraphs for author profiling in social media. In: Mothe et al. [301], pp. 274–280 Rangel Pardo, F., Rosso, P.: On the multilingual and genre robustness of EmoGraphs for author profiling in social media. In: Mothe et al. [301], pp. 274–280
397.
go back to reference Rebholz-Schuhmann, D., et al.: Entity recognition in parallel multi-lingual biomedical corpora: the CLEF-ER laboratory overview. In: Forner et al. [138], pp. 353–367 Rebholz-Schuhmann, D., et al.: Entity recognition in parallel multi-lingual biomedical corpora: the CLEF-ER laboratory overview. In: Forner et al. [138], pp. 353–367
398.
go back to reference Rekabsaz, N., Lupu, M.: A real-world framework for translator as expert retrieval. In: Kanoulas et al. [216], pp. 141–152 Rekabsaz, N., Lupu, M.: A real-world framework for translator as expert retrieval. In: Kanoulas et al. [216], pp. 141–152
399.
go back to reference de Rijke, M., Balog, K., Bogers, T., van den Bosch, A.: On the evaluation of entity profiles. In: Agosti et al. [9], pp. 94–99 de Rijke, M., Balog, K., Bogers, T., van den Bosch, A.: On the evaluation of entity profiles. In: Agosti et al. [9], pp. 94–99
400.
go back to reference Roda, G., Tait, J., Piroi, F., Zenz, V.: CLEF-IP 2009: retrieval experiments in the intellectual property domain. In: Peters et al. [360], pp. 385–409 Roda, G., Tait, J., Piroi, F., Zenz, V.: CLEF-IP 2009: retrieval experiments in the intellectual property domain. In: Peters et al. [360], pp. 385–409
401.
go back to reference Rodrigo, A., Peñas, A., Verdejo, M.F.: Overview of the answer validation exercise 2008. In: Peters et al. [359], pp. 296–313 Rodrigo, A., Peñas, A., Verdejo, M.F.: Overview of the answer validation exercise 2008. In: Peters et al. [359], pp. 296–313
402.
go back to reference Roller, R., Stevenson, M.: Self-supervised relation extraction using UMLS. In: Kanoulas et al. [216], pp. 116–127 Roller, R., Stevenson, M.: Self-supervised relation extraction using UMLS. In: Kanoulas et al. [216], pp. 116–127
403.
go back to reference Rosso, P., et al.: Evolution of the PAN lab on digital text forensics. In: Ferro and Peters [130] Rosso, P., et al.: Evolution of the PAN lab on digital text forensics. In: Ferro and Peters [130]
404.
go back to reference Rosso, P., Rangel, F., Potthast, M., Stamatatos, E., Tschuggnall, M., Stein, B.: Overview of PAN’16. In: Fuhr et al. [143], pp. 332–350 Rosso, P., Rangel, F., Potthast, M., Stamatatos, E., Tschuggnall, M., Stein, B.: Overview of PAN’16. In: Fuhr et al. [143], pp. 332–350
406.
go back to reference Sabetghadam, S., Bierig, R., Rauber, A.: A hybrid approach for multi-faceted IR in multimodal domain. In: Kanoulas et al. [216], pp. 86–97 Sabetghadam, S., Bierig, R., Rauber, A.: A hybrid approach for multi-faceted IR in multimodal domain. In: Kanoulas et al. [216], pp. 86–97
407.
go back to reference Sakai, T.: How to run an evaluation task. In: Ferro and Peters [130] Sakai, T.: How to run an evaluation task. In: Ferro and Peters [130]
408.
go back to reference Saleh, S., Pecina, P.: Reranking hypotheses of machine-translated queries for cross-lingual information retrieval. In: Fuhr et al. [143], pp. 54–68 Saleh, S., Pecina, P.: Reranking hypotheses of machine-translated queries for cross-lingual information retrieval. In: Fuhr et al. [143], pp. 54–68
409.
go back to reference Samuel, J.: Analyzing and visualizing translation patterns of Wikidata properties. In: Bellot et al. [42], pp. 128–134 Samuel, J.: Analyzing and visualizing translation patterns of Wikidata properties. In: Bellot et al. [42], pp. 128–134
410.
go back to reference Sánchez-Junquera, J., Villaseñor-Pineda, L., Montes-y Gómez, M., Rosso, P.: Character N-grams for detecting deceptive controversial opinions. In: Bellot et al. [42], pp. 135–140 Sánchez-Junquera, J., Villaseñor-Pineda, L., Montes-y Gómez, M., Rosso, P.: Character N-grams for detecting deceptive controversial opinions. In: Bellot et al. [42], pp. 135–140
411.
go back to reference Sanchez-Perez, M.A., Gelbukh, A.F., Sidorov, G.: Adaptive algorithm for plagiarism detection: the best-performing approach at PAN 2014 text alignment competition. In: Mothe et al. [301], pp. 402–413 Sanchez-Perez, M.A., Gelbukh, A.F., Sidorov, G.: Adaptive algorithm for plagiarism detection: the best-performing approach at PAN 2014 text alignment competition. In: Mothe et al. [301], pp. 402–413
412.
go back to reference Sanchez-Perez, M.A., Markov, I., Gómez-Adorno, H., Sidorov, G.: Comparison of character n-grams and lexical features on author, gender, and language variety identification on the same Spanish news corpus. In: Jones et al. [209], pp. 145–151 Sanchez-Perez, M.A., Markov, I., Gómez-Adorno, H., Sidorov, G.: Comparison of character n-grams and lexical features on author, gender, and language variety identification on the same Spanish news corpus. In: Jones et al. [209], pp. 145–151
413.
go back to reference SanJuan, E., Moriceau, V., Tannier, X., Bellot, P., Mothe, J.: Overview of the INEX 2012 tweet contextualization track. In: Forner et al. [137] SanJuan, E., Moriceau, V., Tannier, X., Bellot, P., Mothe, J.: Overview of the INEX 2012 tweet contextualization track. In: Forner et al. [137]
414.
go back to reference Santos, D., Cabral, L.M.: GikiCLEF: expectations and lessons learned. In: Peters et al. [360], pp. 212–222 Santos, D., Cabral, L.M.: GikiCLEF: expectations and lessons learned. In: Peters et al. [360], pp. 212–222
415.
go back to reference Ramos dos Santos, W., Paraboni, I.: Personality facets recognition from text. In: Crestani et al. [94] Ramos dos Santos, W., Paraboni, I.: Personality facets recognition from text. In: Crestani et al. [94]
416.
go back to reference Savenkov, D., Braslavski, P., Lebedev, M.: Search snippet evaluation at Yandex: lessons learned and future directions. In: Forner et al. [136], pp. 14–25 Savenkov, D., Braslavski, P., Lebedev, M.: Search snippet evaluation at Yandex: lessons learned and future directions. In: Forner et al. [136], pp. 14–25
417.
go back to reference Savoy, J., Braschler, M.: Lessons learnt from experiments on the ad-hoc multilingual test collections at CLEF. In: Ferro and Peters [130] Savoy, J., Braschler, M.: Lessons learnt from experiments on the ad-hoc multilingual test collections at CLEF. In: Ferro and Peters [130]
418.
go back to reference Schaer, P.: Better than their reputation? On the reliability of relevance assessments with students. In: Catarci et al. [75], pp. 124–135 Schaer, P.: Better than their reputation? On the reliability of relevance assessments with students. In: Catarci et al. [75], pp. 124–135
419.
go back to reference Schaer, P., Mayr, P., Sünkler, S., Lewandowski, D.: How relevant is the long tail?: a relevance assessment study on million short. In: Fuhr et al. [143], pp. 227–233 Schaer, P., Mayr, P., Sünkler, S., Lewandowski, D.: How relevant is the long tail?: a relevance assessment study on million short. In: Fuhr et al. [143], pp. 227–233
420.
go back to reference Schaer, P., Neumann, M.: Enriching existing test collections with OXPath. In: Jones et al. [209], pp. 152–158 Schaer, P., Neumann, M.: Enriching existing test collections with OXPath. In: Jones et al. [209], pp. 152–158
421.
go back to reference Schaüble, P., Sheridan, P.: Cross-language information retrieval (CLIR) track overview. In: Voorhees, E.M., Harman, D.K. (eds.) The Sixth Text REtrieval Conference (TREC-6), pp. 31–44. National Institute of Standards and Technology (NIST), Special Publication 500-240, Washington, USA (1997) Schaüble, P., Sheridan, P.: Cross-language information retrieval (CLIR) track overview. In: Voorhees, E.M., Harman, D.K. (eds.) The Sixth Text REtrieval Conference (TREC-6), pp. 31–44. National Institute of Standards and Technology (NIST), Special Publication 500-240, Washington, USA (1997)
422.
go back to reference Schubotz, M., Krämer, L., Meuschke, N., Hamborg, F., Gipp, B.: Evaluating and improving the extraction of mathematical identifier definitions. In: Jones et al. [209], pp. 82–94 Schubotz, M., Krämer, L., Meuschke, N., Hamborg, F., Gipp, B.: Evaluating and improving the extraction of mathematical identifier definitions. In: Jones et al. [209], pp. 82–94
423.
go back to reference Schuth, A., Balog, K., Kelly, L.: Overview of the living labs for information retrieval evaluation (LL4IR) CLEF Lab 2015. In: Mothe et al. [301], pp. 484–496 Schuth, A., Balog, K., Kelly, L.: Overview of the living labs for information retrieval evaluation (LL4IR) CLEF Lab 2015. In: Mothe et al. [301], pp. 484–496
424.
go back to reference Schuth, A., Marx, M.: Evaluation methods for rankings of facetvalues for faceted search. In: Forner et al. [136], pp. 131–136 Schuth, A., Marx, M.: Evaluation methods for rankings of facetvalues for faceted search. In: Forner et al. [136], pp. 131–136
425.
go back to reference Shen, W., Nie, J.Y.: Is concept mapping useful for biomedical information retrieval. In: Mothe et al. [301], pp. 281–286 Shen, W., Nie, J.Y.: Is concept mapping useful for biomedical information retrieval. In: Mothe et al. [301], pp. 281–286
426.
go back to reference Shepeleva, N., Balog, K.: Towards an understanding of transactional tasks. In: Fuhr et al. [143], pp. 234–240 Shepeleva, N., Balog, K.: Towards an understanding of transactional tasks. In: Fuhr et al. [143], pp. 234–240
427.
go back to reference Sherkat, E., Velcin, J., Milios, E.E.: Fast and simple deterministic seeding of KMeans for text document clustering. In: Bellot et al. [42], pp. 76–88 Sherkat, E., Velcin, J., Milios, E.E.: Fast and simple deterministic seeding of KMeans for text document clustering. In: Bellot et al. [42], pp. 76–88
428.
go back to reference Shing, H.S., Barrow, J., Galuščáková, P., Oard, D.W., Resnik, P.: Unsupervised system combination for set-based retrieval with expectation maximization. In: Crestani et al. [94] Shing, H.S., Barrow, J., Galuščáková, P., Oard, D.W., Resnik, P.: Unsupervised system combination for set-based retrieval with expectation maximization. In: Crestani et al. [94]
429.
go back to reference Sierek, T., Hanbury, A.: Using health statistics to improve medical and health search. In: Mothe et al. [301], pp. 287–292 Sierek, T., Hanbury, A.: Using health statistics to improve medical and health search. In: Mothe et al. [301], pp. 287–292
430.
go back to reference Sigurbjörnsson, B., Kamps, J., de Rijke, M.: Overview of WebCLEF 2005. In: Peters et al. [365], pp. 810–824 Sigurbjörnsson, B., Kamps, J., de Rijke, M.: Overview of WebCLEF 2005. In: Peters et al. [365], pp. 810–824
431.
go back to reference Silvello, G., Bordea, G., Ferro, N., Buitelaar, P., Bogers, T.: Semantic representation and enrichment of information retrieval experimental data. Int. J. Digit. Libr. (IJDL) 18(2), 145–172 (2017)CrossRef Silvello, G., Bordea, G., Ferro, N., Buitelaar, P., Bogers, T.: Semantic representation and enrichment of information retrieval experimental data. Int. J. Digit. Libr. (IJDL) 18(2), 145–172 (2017)CrossRef
432.
go back to reference Singh, G., Mantrach, A., Silvestri, F.: Improving profiles of weakly-engaged users: with applications to recommender systems. In: Fuhr et al. [143], pp. 123–136 Singh, G., Mantrach, A., Silvestri, F.: Improving profiles of weakly-engaged users: with applications to recommender systems. In: Fuhr et al. [143], pp. 123–136
433.
go back to reference Sorg, P., Cimiano, P., Schultz, A., Sizov, S.: Overview of the cross-lingual expert search (CriES) pilot challenge. In: Braschler et al. [61] Sorg, P., Cimiano, P., Schultz, A., Sizov, S.: Overview of the cross-lingual expert search (CriES) pilot challenge. In: Braschler et al. [61]
434.
go back to reference Spina, D., Amigó, E., Gonzalo, J.: Filter keywords and majority class strategies for company name disambiguation in Twitter. In: Forner et al. [136], pp. 38–49 Spina, D., Amigó, E., Gonzalo, J.: Filter keywords and majority class strategies for company name disambiguation in Twitter. In: Forner et al. [136], pp. 38–49
435.
go back to reference Stamatatos, E., Potthast, M., Rangel Pardo, F.M., Rosso, P., Stein, B.: Overview of the PAN/CLEF 2015 evaluation lab. In: Mothe et al. [301], pp. 518–538 Stamatatos, E., Potthast, M., Rangel Pardo, F.M., Rosso, P., Stein, B.: Overview of the PAN/CLEF 2015 evaluation lab. In: Mothe et al. [301], pp. 518–538
436.
go back to reference Stamatatos, E., et al.: Overview of PAN 2018. In: Bellot et al. [42], pp. 267–285 Stamatatos, E., et al.: Overview of PAN 2018. In: Bellot et al. [42], pp. 267–285
437.
go back to reference Stathopoulos, S., Kalamboukis, T.: Medical image classification with weighted latent semantic tensors and deep convolutional neural networks. In: Bellot et al. [42], pp. 89–100 Stathopoulos, S., Kalamboukis, T.: Medical image classification with weighted latent semantic tensors and deep convolutional neural networks. In: Bellot et al. [42], pp. 89–100
438.
go back to reference Stefanov, V., Sachs, A., Kritz, M., Samwald, M., Gschwandtner, M., Hanbury, A.: A formative evaluation of a comprehensive search system for medical professionals. In: Forner et al. [138], pp. 81–92 Stefanov, V., Sachs, A., Kritz, M., Samwald, M., Gschwandtner, M., Hanbury, A.: A formative evaluation of a comprehensive search system for medical professionals. In: Forner et al. [138], pp. 81–92
439.
go back to reference Stempfhuber, M., Baerisch, S.: The domain-specific track at CLEF 2006: overview of approaches, results and assessment. In: Peters et al. [357], pp. 163–169 Stempfhuber, M., Baerisch, S.: The domain-specific track at CLEF 2006: overview of approaches, results and assessment. In: Peters et al. [357], pp. 163–169
440.
go back to reference Suchomel, S., Brandejs, M.: Determining window size from plagiarism corpus for stylometric features. In: Mothe et al. [301], pp. 293–299 Suchomel, S., Brandejs, M.: Determining window size from plagiarism corpus for stylometric features. In: Mothe et al. [301], pp. 293–299
441.
go back to reference Suominen, H.: CLEFeHealth2012 - the CLEF 2012 workshop on cross-language evaluation of methods, applications, and resources for eHealth document analysis. In: Forner et al. [137] Suominen, H.: CLEFeHealth2012 - the CLEF 2012 workshop on cross-language evaluation of methods, applications, and resources for eHealth document analysis. In: Forner et al. [137]
442.
go back to reference Suominen, H., Kelly, L., Goeuriot, L.: The scholarly impact and strategic intent of CLEF eHealth labs from 2012–2017. In: Ferro and Peters [130] Suominen, H., Kelly, L., Goeuriot, L.: The scholarly impact and strategic intent of CLEF eHealth labs from 2012–2017. In: Ferro and Peters [130]
443.
go back to reference Suominen, H., et al.: Overview of the CLEF eHealth evaluation lab 2018. In: Bellot et al. [42], pp. 286–301 Suominen, H., et al.: Overview of the CLEF eHealth evaluation lab 2018. In: Bellot et al. [42], pp. 286–301
444.
go back to reference Suominen, H., et al.: Overview of the CLEF eHealth evaluation lab 2019. In: Crestani et al. [94] Suominen, H., et al.: Overview of the CLEF eHealth evaluation lab 2019. In: Crestani et al. [94]
445.
go back to reference Suominen, H., et al.: Overview of the ShARe/CLEF eHealth evaluation lab 2013. In: Forner et al. [138], pp. 212–231 Suominen, H., et al.: Overview of the ShARe/CLEF eHealth evaluation lab 2013. In: Forner et al. [138], pp. 212–231
446.
go back to reference Tannebaum, W., Mahdabi, P., Rauber, A.: Effect of log-based query term expansion on retrieval effectiveness in patent searching. In: Mothe et al. [301], pp. 300–305 Tannebaum, W., Mahdabi, P., Rauber, A.: Effect of log-based query term expansion on retrieval effectiveness in patent searching. In: Mothe et al. [301], pp. 300–305
447.
go back to reference Tannebaum, W., Rauber, A.: Mining query logs of USPTO patent examiners. In: Forner et al. [138], pp. 136–142 Tannebaum, W., Rauber, A.: Mining query logs of USPTO patent examiners. In: Forner et al. [138], pp. 136–142
448.
go back to reference Teixeira Lopes, C., Almeida Fernandes, T.: Health suggestions: a chrome extension to help laypersons search for health information. In: Fuhr et al. [143], pp. 241–246 Teixeira Lopes, C., Almeida Fernandes, T.: Health suggestions: a chrome extension to help laypersons search for health information. In: Fuhr et al. [143], pp. 241–246
449.
go back to reference Teixeira Lopes, C., Ribeiro, C.: Effects of language and terminology on the usage of health query suggestions. In: Fuhr et al. [143], pp. 83–95 Teixeira Lopes, C., Ribeiro, C.: Effects of language and terminology on the usage of health query suggestions. In: Fuhr et al. [143], pp. 83–95
450.
go back to reference Teixeira Lopes, C., Ribeiro, C.: Effects of language and terminology of query suggestions on the precision of health searches. In: Bellot et al. [42], pp. 101–111 Teixeira Lopes, C., Ribeiro, C.: Effects of language and terminology of query suggestions on the precision of health searches. In: Bellot et al. [42], pp. 101–111
451.
go back to reference Thomee, B., Popescu, A.: Overview of the ImageCLEF 2012 Flickr photo annotation and retrieval task. In: Forner et al. [137] Thomee, B., Popescu, A.: Overview of the ImageCLEF 2012 Flickr photo annotation and retrieval task. In: Forner et al. [137]
452.
go back to reference Thornley, C.V., Johnson, A.C., Smeaton, A.F., Lee, H.: The scholarly impact of TRECVid (2003–2009). J. Am. Soc. Inf. Sci. Technol. (JASIST) 62(4), 613–627 (2011)CrossRef Thornley, C.V., Johnson, A.C., Smeaton, A.F., Lee, H.: The scholarly impact of TRECVid (2003–2009). J. Am. Soc. Inf. Sci. Technol. (JASIST) 62(4), 613–627 (2011)CrossRef
453.
go back to reference Tommasi, T., Caputo, B., Welter, P., Güld, M.O., Deserno, T.M.: Overview of the CLEF 2009 medical image annotation track. In: Peters et al. [368], pp. 85–93 Tommasi, T., Caputo, B., Welter, P., Güld, M.O., Deserno, T.M.: Overview of the CLEF 2009 medical image annotation track. In: Peters et al. [368], pp. 85–93
454.
go back to reference Trappett, M., Geva, S., Trotman, A., Scholer, F., Sanderson, M.: Overview of the INEX 2012 snippet retrieval track. In: Forner et al. [137] Trappett, M., Geva, S., Trotman, A., Scholer, F., Sanderson, M.: Overview of the INEX 2012 snippet retrieval track. In: Forner et al. [137]
455.
go back to reference Trotzek, M., Koitka, S., Friedrich, C.M.: Early detection of depression based on linguistic metadata augmented classifiers revisited. In: Bellot et al. [42], pp. 191–202 Trotzek, M., Koitka, S., Friedrich, C.M.: Early detection of depression based on linguistic metadata augmented classifiers revisited. In: Bellot et al. [42], pp. 191–202
456.
go back to reference Tsikrika, T., Garcia Seco de Herrera, A., Müller, H.: Assessing the scholarly impact of ImageCLEF. In: Forner et al. [136], pp. 95–106 Tsikrika, T., Garcia Seco de Herrera, A., Müller, H.: Assessing the scholarly impact of ImageCLEF. In: Forner et al. [136], pp. 95–106
457.
go back to reference Tsikrika, T., Kludas, J.: Overview of the WikipediaMM task at ImageCLEF 2008. In: Peters et al. [359], pp. 539–550 Tsikrika, T., Kludas, J.: Overview of the WikipediaMM task at ImageCLEF 2008. In: Peters et al. [359], pp. 539–550
458.
go back to reference Tsikrika, T., Kludas, J.: Overview of the WikipediaMM task at ImageCLEF 2009. In: Peters et al. [368], pp. 60–71 Tsikrika, T., Kludas, J.: Overview of the WikipediaMM task at ImageCLEF 2009. In: Peters et al. [368], pp. 60–71
459.
go back to reference Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Forner et al. [138], pp. 1–12 Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Forner et al. [138], pp. 1–12
460.
go back to reference Tsikrika, T., Popescu, A., Kludas, J.: Overview of the Wikipedia image retrieval task at ImageCLEF 2011. In: Petras et al. [373] Tsikrika, T., Popescu, A., Kludas, J.: Overview of the Wikipedia image retrieval task at ImageCLEF 2011. In: Petras et al. [373]
461.
go back to reference Turchi, M., Steinberger, J., Alexandrov Kabadjov, M., Steinberger, R.: Using parallel corpora for multilingual (multi-document) summarisation evaluation. In: Agosti et al. [9], pp. 52–63 Turchi, M., Steinberger, J., Alexandrov Kabadjov, M., Steinberger, R.: Using parallel corpora for multilingual (multi-document) summarisation evaluation. In: Agosti et al. [9], pp. 52–63
462.
go back to reference Turmo, J., et al.: Overview of QAST 2009. In: Peters et al. [360], pp. 197–211 Turmo, J., et al.: Overview of QAST 2009. In: Peters et al. [360], pp. 197–211
463.
go back to reference Turmo, J., Comas, P., Rosset, S., Lamel, L., Moreau, N., Mostefa, D.: Overview of QAST 2008. In: Peters et al. [359], pp. 296–313 Turmo, J., Comas, P., Rosset, S., Lamel, L., Moreau, N., Mostefa, D.: Overview of QAST 2008. In: Peters et al. [359], pp. 296–313
464.
go back to reference Vallin, A., et al.: Overview of the CLEF 2005 multilingual question answering track. In: Peters et al. [365], pp. 307–331 Vallin, A., et al.: Overview of the CLEF 2005 multilingual question answering track. In: Peters et al. [365], pp. 307–331
465.
go back to reference Valverde-Albacete, F., Carrillo de Albornoz, J., Peláez-Moreno, C.: A proposal for new evaluation metrics and result visualization technique for sentiment analysis tasks. In: Forner et al. [138], pp. 41–42 Valverde-Albacete, F., Carrillo de Albornoz, J., Peláez-Moreno, C.: A proposal for new evaluation metrics and result visualization technique for sentiment analysis tasks. In: Forner et al. [138], pp. 41–42
466.
go back to reference Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2015 labs. In: Mothe et al. [301], pp. 444–461 Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2015 labs. In: Mothe et al. [301], pp. 444–461
467.
go back to reference Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2016 labs. In: Fuhr et al. [143], pp. 267–285 Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2016 labs. In: Fuhr et al. [143], pp. 267–285
468.
go back to reference Villegas, M., Paredes, R.: Overview of the ImageCLEF 2012 scalable web image annotation task. In: Forner et al. [137] Villegas, M., Paredes, R.: Overview of the ImageCLEF 2012 scalable web image annotation task. In: Forner et al. [137]
469.
go back to reference Voorhees, E.M.: TREC: continuing information retrieval’s tradition of experimentation. Commun. ACM (CACM) 50(11), 51–54 (2007)CrossRef Voorhees, E.M.: TREC: continuing information retrieval’s tradition of experimentation. Commun. ACM (CACM) 50(11), 51–54 (2007)CrossRef
470.
go back to reference Voorhees, E.M.: The evolution of cranfield. In: Ferro and Peters [130] Voorhees, E.M.: The evolution of cranfield. In: Ferro and Peters [130]
471.
go back to reference Wakeling, S., Clough, P.: Integrating mixed-methods for evaluating information access systems. In: Mothe et al. [301], pp. 306–311 Wakeling, S., Clough, P.: Integrating mixed-methods for evaluating information access systems. In: Mothe et al. [301], pp. 306–311
472.
go back to reference Walker, A., Starkey, A., Pan, J.Z., Siddharthan, A.: Making test corpora for question answering more representative. In: Kanoulas et al. [216], pp. 1–6 Walker, A., Starkey, A., Pan, J.Z., Siddharthan, A.: Making test corpora for question answering more representative. In: Kanoulas et al. [216], pp. 1–6
473.
go back to reference Wang, J., Gilbert, A., Thomee, B., Villegas, M.: Automatic image annotation at ImageCLEF. In: Ferro and Peters [130] Wang, J., Gilbert, A., Thomee, B., Villegas, M.: Automatic image annotation at ImageCLEF. In: Ferro and Peters [130]
474.
go back to reference Wang, Q., et al.: Overview of the INEX 2012 linked data track. In: Forner et al. [137] Wang, Q., et al.: Overview of the INEX 2012 linked data track. In: Forner et al. [137]
475.
go back to reference Wang, X., Guo, Z., Zhang, Y., Li, J.: Medical image labeling and semantic understanding for clinical applications. In: Crestani et al. [94] Wang, X., Guo, Z., Zhang, Y., Li, J.: Medical image labeling and semantic understanding for clinical applications. In: Crestani et al. [94]
476.
go back to reference Wang, X., Wang, X., Zhang, Q.: A web-based CLIR system with cross-lingual topical pseudo relevance feedback. In: Forner et al. [138], pp. 104–107 Wang, X., Wang, X., Zhang, Q.: A web-based CLIR system with cross-lingual topical pseudo relevance feedback. In: Forner et al. [138], pp. 104–107
477.
go back to reference Weitzel, L., Bernardini, F., Quaresma, P., Alves, C.A., Zacharski, W., de Figueiredo, L.G.: Brazilian social mood: the political dimension of emotion. In: Fuhr et al. [143], pp. 247–255 Weitzel, L., Bernardini, F., Quaresma, P., Alves, C.A., Zacharski, W., de Figueiredo, L.G.: Brazilian social mood: the political dimension of emotion. In: Fuhr et al. [143], pp. 247–255
478.
go back to reference White, R.W., Oard, D.W., Jones, G.J.F., Soergel, D., Huang, X.: Overview of the CLEF-2005 cross-language speech retrieval track. In: Peters et al. [365], pp. 744–759 White, R.W., Oard, D.W., Jones, G.J.F., Soergel, D., Huang, X.: Overview of the CLEF-2005 cross-language speech retrieval track. In: Peters et al. [365], pp. 744–759
479.
go back to reference Wilhelm-Stein, T., Eibl, M.: A quantitative look at the CLEF working notes. In: Forner et al. [138], pp. 13–16 Wilhelm-Stein, T., Eibl, M.: A quantitative look at the CLEF working notes. In: Forner et al. [138], pp. 13–16
480.
go back to reference Wilhelm-Stein, T., Eibl, M.: Teaching the IR process using real experiments supported by game mechanics. In: Mothe et al. [301], pp. 312–317 Wilhelm-Stein, T., Eibl, M.: Teaching the IR process using real experiments supported by game mechanics. In: Mothe et al. [301], pp. 312–317
481.
go back to reference Wilhelm-Stein, T., Herms, R., Ritter, M., Eibl, M.: Improving transcript-based video retrieval using unsupervised language model adaptation. In: Kanoulas et al. [216], pp. 110–115 Wilhelm-Stein, T., Herms, R., Ritter, M., Eibl, M.: Improving transcript-based video retrieval using unsupervised language model adaptation. In: Kanoulas et al. [216], pp. 110–115
482.
go back to reference Xu, K., Feng, Y., Huang, S., Zhao, D.: Question answering via phrasal semantic parsing. In: Mothe et al. [301], pp. 414–426 Xu, K., Feng, Y., Huang, S., Zhao, D.: Question answering via phrasal semantic parsing. In: Mothe et al. [301], pp. 414–426
483.
go back to reference Yan, X., Gao, G., Su, X., Wei, H., Zhang, X., Lu, Q.: Hidden Markov model for term weighting in verbose queries. In: Catarci et al. [75], pp. 82–87 Yan, X., Gao, G., Su, X., Wei, H., Zhang, X., Lu, Q.: Hidden Markov model for term weighting in verbose queries. In: Catarci et al. [75], pp. 82–87
484.
go back to reference Yang, H., Gonçalves, T.: A compound model for consumer health search. In: Bellot et al. [42], pp. 231–236 Yang, H., Gonçalves, T.: A compound model for consumer health search. In: Bellot et al. [42], pp. 231–236
485.
go back to reference Zamani, H., Esfahani, H.N., Babaie, P., Abnar, S., Dehghani, M., Shakery, A.: Authorship identification using dynamic selection of features from probabilistic feature set. In: Kanoulas et al. [216], pp. 128–140 Zamani, H., Esfahani, H.N., Babaie, P., Abnar, S., Dehghani, M., Shakery, A.: Authorship identification using dynamic selection of features from probabilistic feature set. In: Kanoulas et al. [216], pp. 128–140
486.
go back to reference Zellhöfer, D.: Overview of the personal photo retrieval pilot task at ImageCLEF 2012. In: Forner et al. [137] Zellhöfer, D.: Overview of the personal photo retrieval pilot task at ImageCLEF 2012. In: Forner et al. [137]
487.
go back to reference Zhang, L., Rettinger, A., Färber, M., Tadic, M.: A comparative evaluation of cross-lingual text annotation techniques. In: Forner et al. [138], pp. 124–135 Zhang, L., Rettinger, A., Färber, M., Tadic, M.: A comparative evaluation of cross-lingual text annotation techniques. In: Forner et al. [138], pp. 124–135
488.
go back to reference Ziak, H., Kern, R.: Evaluation of pseudo relevance feedback techniques for cross vertical aggregated search. In: Mothe et al. [301], pp. 91–102 Ziak, H., Kern, R.: Evaluation of pseudo relevance feedback techniques for cross vertical aggregated search. In: Mothe et al. [301], pp. 91–102
489.
go back to reference Zingla, M.A., Latiri, C., Slimani, Y.: Tweet contextualization using association rules mining and DBpedia. In: Mothe et al. [301], pp. 318–323 Zingla, M.A., Latiri, C., Slimani, Y.: Tweet contextualization using association rules mining and DBpedia. In: Mothe et al. [301], pp. 318–323
490.
go back to reference Zlabinger, M., Rekabsaz, N., Zlabinger, S., Hanbury, A.: Efficient answer-annotation for frequent questions. In: Crestani et al. [94] Zlabinger, M., Rekabsaz, N., Zlabinger, S., Hanbury, A.: Efficient answer-annotation for frequent questions. In: Crestani et al. [94]
491.
go back to reference Zuo, C., Karakas, A., Banerjee, R.: To check or not to check: syntax, semantics, and context in the language of check-worthy claims. In: Crestani et al. [94] Zuo, C., Karakas, A., Banerjee, R.: To check or not to check: syntax, semantics, and context in the language of check-worthy claims. In: Crestani et al. [94]
Metadata
Title
What Happened in CLEF For a While?
Author
Nicola Ferro
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-28577-7_1

Premium Partner