Overview of the first TREC conference

ABSTRACT

The first Text REtrieval Conference (TREC-1) was held in early November 1992 and was attended by about 100 people working in the 25 participating groups. The goal of the conference was to bring research groups together to discuss their work on a new large test collection. There was a large variety of retrieval techniques reported on, including methods using automatic thesaurii, sophisticated term weighting, natural language techniques, relevance feedback, and advanced pattern matching. As results had been run through a common evaluation package, groups were able to compare the effectiveness of different techniques, and discuss how differences among the sytems affected performance.

REFERENCES

• 1: Buckley C., Salton G. & Allan L (1993). Automatic Retrieval with Locality Information Using SMART. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 59-72). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 2: Chang S-C,, Dc~l{u H., Az~n H, & Du M-W.. (1993). Multilevel Ranking in Large Text Collections Using FAIRS. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 329-336). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 3: Cleverdon C.W., (1962). Report on the Testing and Analysis of an Investigation into the Comparative Efftciency of Indexing Systems. College of Aeronautics, Cranfield, England, 1962.
• 4: Cleverdon C.W., Mills, J. & Keen E.M. (1966). Factors Determining the Performance of Indexing Systems, Vot. 1: Design, Vol. 2: Test Results. Aslib Cranfield Research Project, Cranfield, England, 1966.
• 5: Evans D., Lefferts R., Grefenstette G., Handerson S., Hersh W. & Archbold A.(1993). CLARIT TREC Design, Experiments, and Results. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 251-286). National Institute of Standards and Technology Special Publication 500-207, Gaithersbtrrg, Md. 20899.
• 6: Fox E. (1983). Characteristics of Two New Experimental Collections in Computer and Information Science Contaln.hag Textual and Bibliographic Concepts. Technical Report TR 83-561, Comell University: Computing Science Department.
• 7: Fuhr N. & Buckley C. (1993). Optimizing Document Indexing and Search Term Weighting Based on Probabilisitic Models. In D. Harman (Ed.), The First Text REtrieval Conference (TREC- 1) (,pp. 89-100). Nadonal Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 208'99.
• 8: Donna Harman, The DARPA TIPSTER project, ACM SIGIR Forum, v.26 n.2, p.26-28, Fall 1992 [doi>10.1145/146565.146567]
• 9: Harman D. (Ed.). (1993b). The First Text REtrieval Conference (TREC-1). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 10: Jacobs P., Krupka G. & Rau L. (1993). A Boolean Approximation Method for Query Construction and Topic Assigrmaent in TREC. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 297-308). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 11: Jones R., Leung S. & Pape D.L. (1993). Application of the Automatic Message Router to the TIPSTER Collection. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-I) (pp. 245-250). National Institute of Standards and Tectmology Special Publication 500-207, Gaithersburg, Md. 20899.
• 12: Katzer J., McGill M.J., Tessier J.A., Frakes W., and D,LsGupta P. (1982). A Study of the Overlap among Document Representations. Information Technology: Research and Development, 1(2), 261-274.
• 13: Kwok K., Papadopoulos L. & Kwan K. (1993). Retrieval Experiments with a Large Collection using PIRCS. In D. Hatman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 153-172). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 14: Metfler M. (1993). Text Retrieval with the TRW Fast Data Finder. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 309-318). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 15: Miller G. (1990). WordNet: An On-line Lexical Database. International Journal of Lexicography, 3(4).
• 16: Nelson P. (1993). Site Report for the Text REtrieval Conference. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 287-296). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 17: Robertson S., Walker S., Hancock-Beaulieu M., Gull A. & Lau M. (1993). Okapi at TREC. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 21-30). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 18: Salton G. (1969). A Comparison Between Manual and Automatic Indexing Methods. American Documentation, 20(1).
• 19: Karen Sparck Jones, Information Retrieval Experiment, Butterworth-Heinemann, Newton, MA, 1981
• 20: Sparck Jones K. and Van Rijsbergen C. (1975). Report on the Need for and Provision of an "Ideal" Information Retrieval Test Collection, British Library Research and Development Report 5266, Computer Laboratory, University of Cambridge.
• 21: Sparck Jones K. and Webster C.A. (1979). Research in Relevance Weighting, British Library Research and Development Report 5553, Computer Laboratory, University of Cambridge.
• 22: Strzalkowski T. (1993). Natural Language Processing in Large-Scale Text Retrieval Tasks. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 173-188). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 23: Voorhees E. & Hou Y-W. (1993). Vector Expansion in a Large Collection. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 343-352). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.
• 24: Zimmerman M. (1993). Proximity-Correlation for Document Ranking: The PARA Group's TREC Experiment. In D. Harman (Ed.), The First Text REtrieval Conference (TREC-1) (pp. 353-364). National Institute of Standards and Technology Special Publication 500-207, Gaithersburg, Md. 20899.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

General Terms:
Algorithms, Design, Human Factors, Performance