Gordon V. Cormack
- 1.C. L. A. Clarke and G. V. Cormack. Interactive substring retrival. In D. K. Harman and E. M. Voorhees, editors, information Technology: The Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, November 1996. National Institute of Standards and Technology (NIST), United States Department of Commerce. Available electronically at http://trec .nist .gov.Google Scholar
- 2.C. L. A. Clarke, G. V. Cormack, and F. J. Burkowski. Shortest substring ranking. In D. K. Harman, editor, The Fourth Text REtrieval Conference (TREC-$), pages 295-304, Gaithersburg, Maryland, November 1995. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-238. Available electronically at http:/#tree, hist. gov.Google Scholar
- 3.G. V. Cormack, C. L. A. Clarke, C. 1t. Palmer, and S. S.-L. To. Passage based refinement. In Sixth Text REtrieval Conference (TREC-6), Gaithersburg, Maryland, November 1997. National Institute of Standards and Technology (NIST), United States Department of Commerce. Available electronically at http://tree .nisz .gov.Google Scholar
- 4.H. Gilbert and K. S. Jones. Statistical bases of relevance assessment for the 'ideal' information retrieval test collection. Technical report, Computer Laboratory, University of Cambridge, 1979. BL R&D Report 5481.Google Scholar
- 5.D. Harman. Overview of the first TREC conference. In 16th Annual International A CM SIGIR Conference on Research and Development in Information Retrieval, pages 36-47, Pittsburgh, PA, june 1993. Google ScholarDigital Library
- 6.D. Harman. Overview of the fourth Text RE- trieval Conference (TREC-4). In D. K. Harman, editor, The Fourth Text REtrieval Conference (TREC- #{), pages 1-23, Gaithersburg, Maryland, November 1995. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-236. Available electronically at http://tree .aist .gov.Google ScholarCross Ref
- 7.D. K. Harman, editor. The First Text REtrieval Conference (TREC-1), Gaithersburg, Maryland, November 1992. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-207.Google Scholar
- 8.D. K. Harman, editor. The Second Text RE- trieval Conference (TREC-#), Gaithersburg, Maryland, November 1993. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-215. Google ScholarDigital Library
- 9.D. K. Harmon, editor. Overview of the Third Text REtrieval Conference (TREC-3), Gaithersburg, Maryland, November 1994. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-225. Available electronically at http://trec, hist. gov.Google Scholar
- 10.D. K. Harmon. Overview of the third Text REtrieval Conference (TREC-3). In D. K. Harmon, editor, Overview of the Third Te#t REtrieval Conference (TREC-3), pages 1-19, Gaithersburg, Maryland, November 1994. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-225. Available electronically at http://trec .hist. gov.Google Scholar
- 11.D. K. Harmon, editor. The Fourth Text REtrieval Conference (TRBC-j), Gaithersburg, Maryland, November 1995. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-236. Available electronically at http://trec .nist. gov.Google Scholar
- 12.D.K. Harman and E. M. Voorhees, editors. Information Technology: The Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, November 1996. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-238. Available electronically at http://trec.nist.gov.Google Scholar
- 13.D. K. Harman and E. M. Voorhees, editors. The Sixth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, November 1997. National Institute of Standards and Technology (NIST), United States Department of Commerce. Available electronically at http://tree .n~st. gov.Google Scholar
- 14.M. E. Lesk and G. Salton. Relevance assessments and retrieval system evaluation. Information Storage and Management, 4:343-359, 1966.Google ScholarCross Ref
- 15.E. V. Paul B. Kantor. Report on the TREC-5 confusion track. In D. K. Harmon, editor, Information Technology: The Fifth Text REtrieval Conference (TREC-5), pages 65-74, Gaithersburg, Maryland, November 1996. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-236. Available electronically at http://trec .nist. gov.Google Scholar
- 16.S. E. Robertson. The probability ranking principle in it. Journal of Documentation, 33:294-304, 1977.Google ScholarCross Ref
- 17.P. Sheridan, J. P. Ballerini, and P. Sch#iuble. Building a large multilingual test collection from comparable news documents. In G. Grefenstette, A. Smeaton, and P. Sheridan, editors, Workshop on Cross-Linguistic Information Retrieval, pages 56- 65. ACM SIGIR, Aug. 1996. Google ScholarDigital Library
- 18.K. Sparck Jones and C. J. Van Rijsbergen. Report on the need for and provision of an 'ideal' test collection. Technical report, University Computer Laboratory, Cambridge, 1975.Google Scholar
- 19.K. Sparck Jones and C. J. Van Rijsbergen. Information retrieval test collections. Journal of Documentation, 32(1):59-72, March 1976.Google ScholarCross Ref
- 20.J. Tague-Sutcliffe and J. Blustein. A statistical analysis of the TREC-3 data. In D. K. Harman, editor, Overview of the Third Text REtrieval Con}erence (TREC-3), pages 385-398, Gaithersburg, Maryland, November 1994. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-225. Available electronically at http://trec.nist, gov.Google Scholar
- 21.E. M. Voorhees. Variations in relevance judgements and the measurement of retrieval effectiveness. In #lst Annual International A CM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, August 1998. Google ScholarDigital Library
- 22.N. West. Applied Statistics for Marine Affairs Professionals. Praeger, Westport, CT, 1996.Google Scholar
- 23.J. Zobel. How reliable are the results of large-scale information retrieval experiments? In Zlst Annual International A CM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, August 1998. Google ScholarDigital Library
Index Terms
- Efficient construction of large test collections
Recommendations
On the Reusability of Personalized Test Collections
UMAP '17: Adjunct Publication of the 25th Conference on User Modeling, Adaptation and PersonalizationTest collections for offline evaluation remain crucial for information retrieval research and industrial practice, yet reusability of test collections is under threat by different factors such as dynamic nature of data collections and new trends in ...
Test theory for assessing IR test collections
SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrievalHow good is an IR test collection? A series of papers in recent years has addressed the question by empirically enumerating the consistency of performance comparisons using alternate subsets of the collection. In this paper we propose using Test Theory, ...
Building Test Collections: An Interactive Guide for Students and Others Without Their Own Evaluation Conference Series
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information RetrievalThis is a full-day tutorial on building and validating test collections. The intended audience is advanced students who nd themselves in need of a test collection, or actually in the process of building a test collection, to support their own research. ...
Comments