Concept based query expansion

ABSTRACT

Query expansion methods have been studied for a long time - with debatable success in many instances. In this paper we present a probabilistic query expansion model based on a similarity thesaurus which was constructed automatically. A similarity thesaurus reflects domain knowledge about the particular collection from which it is constructed. We address the two important issues with query expansion: the selection and the weighting of additional search terms. In contrast to earlier methods, our queries are expanded by adding those terms that are most similar to the concept of the query, rather than selecting terms that are similar to the query terms. Our experiments show that this kind of query expansion results in a notable improvement in the retrieval effectiveness when measured using both recall-precision and usefulness.

REFERENCES

• Cro 90: C. J. Crouch, An approach to the automatic construction of global thesauri, Information Processing and Management: an International Journal, v.26 n.5, p.629-640, 1990 [doi>10.1016/0306-4573(90)90106-C]
• Cro 92: Carolyn J. Crouch , Bokyung Yang, Experiments in automatic statistical thesaurus construction, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.77-88, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133180]
• Dee 90: Deerwester, S., Dumais, S.T., Fumas, G.W., Landauer, T.K., Harshman R., Indexing by latent semantic analysis, J. of the ASIS, 41 (6): 391-407, 1990.
• Ekm 92: F. Cuna Ekmekcioglu , Alexander M. Robertson , Peter Willett, Effectiveness of query expansion in ranked-output document retrieval systems, Journal of Information Science, v.18 n.2, p.139-147, 1992 [doi>10.1177/016555159201800208]
• Fre 91: H. P. Frei , P. Schäuble, Determining the effectiveness of retrieval algorithms, Information Processing and Management: an International Journal, v.27 n.2-3, p.153-164, Feb./March 1991 [doi>10.1016/0306-4573(91)90046-O]
• Gre 92: Gregory Grefenstette, Use of syntactic context to produce term association lists for text retrieval, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.89-97, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133181]
• Han 92: Micheline Hancock-Beaulieu, Query expansion: advances in research in online catalogues, Journal of Information Science, v.18 n.2, p.99-103, 1992 [doi>10.1177/016555159201800203]
• Har92: Donna Harman, Relevance feedback revisited, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.1-10, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133167]
• Les 69: Lesk, M.E., Word-word association in document retrieval systems, American Documentation, 20(1): 27-38, 1969.
• Min 72: Minker, J., Wilson, G.A., Zirnmerman, B.I-I., An evaluation of query expansion by the addition of clustered terms for a document retrieval system, Information Storage and Retrieval, 8(6): 329-48, 1972.
• Pea 91: Peat, H.J., Willett, P., The limitations of term co-occurrence data for query expansion in document retrieval systems, J. of the ASIS, 42(5): 378-83, 1991.
• Qiu 92: Qiu, Y., ISIR: an integrated system for information retrieval, Proc. 14th 1R Colloquium, British Computer Society, Lancaster, 1992.
• Rug 92: Gerda Ruge, Experiment on linguistically-based term associations, Information Processing and Management: an International Journal, v.28 n.3, p.317-332, 1992 [doi>10.1016/0306-4573(92)90078-E]
• Sal 71: Salton, G., Experiments in automatic thesaurus construction for information retrieval, Information Processing 71, 1: 115-123, 1971.
• Sal 75: Salton, G., Yang, C.S., Yu, C.T., A theory of term importance in automatic text analysis, J. of the ASIS, 26(1): 33-44, 1975.
• Sal 80: Salton, G., Automatic term class construction using relevance-a summary of work in automatic pseudoclassification, Information Processing & Management, 16(1): 1-15, 1980.
• Sal 88: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
• Sal 90: Salton, G., Buckley, C.: Improving Retrieval Performance by Relevance Feedback. J. of the ASIS, 41(4): 288-297, 1990.
• Sch 92: Sch~iuble, P., Knaus, D., The various roles of information structures, 16. Jahrestagung der Gesellschaft ffir Klassifikation, Dortmund, 1992.
• Sme 83: Smeaton, A.F., van Rijsbergen, C.J., The retrieval effects of query expansion on a feedback document retrieval system, The Computer Journal, 26(3): 239-46, 1983.
• Spa 71: Sparck-Jones, K., Barber, E.B., What makes an automatic keyword classification effective? J. of the ASIS, 18." 166-175, 1971.
• Spa 91: Karen Sparck Jones, Notes and references on early automatic classification work, ACM SIGIR Forum, v.25 n.1, p.10-17, Spring 1991 [doi>10.1145/122642.122644]
• Wad 88: Stephe J. Wade , Peter Willett, INSTRUCT: a teaching package for experimental methods in information retrieval. Part 111. Browsing, clustering and query, Program, v.22 n.1, p.44-61, January 1988 [doi>10.1108/eb046986]
• Won 87: S. K.M. Wong , W. Ziarko , V. V. Raghavan , P. C.N. Wong, On modeling of information retrieval concepts in vector spaces, ACM Transactions on Database Systems (TODS), v.12 n.2, p.299-321, June 1987 [doi>10.1145/22952.22957]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods; Thesauruses

General Terms:
Algorithms, Experimentation, Languages