Quantify query ambiguity using ODP metadata

ABSTRACT

Query ambiguity prevents existing retrieval systems from returning reasonable results for every query. As there is already lots of work done on resolving ambiguity, vague queries could be handled using corresponding approaches separately if they can be identified in advance. Quantification of the degree of (lack of) ambiguity laysthe groundwork for the identification. In this poster, we propose such a measure using query topics based on the topic structure selected from the Open Directory Project (ODP) taxonomy. We introduce clarity score to quantify the lack of ambiguity with respect to data sets constructed from the TREC collections and the rank correlation test results demonstrate a strong positive association between the clarity scores and retrieval precisions for queries.

REFERENCES

• 1: Steve Cronen-Townsend , W. Bruce Croft, Quantifying query ambiguity, Proceedings of the second international conference on Human Language Technology Research, March 24-27, 2002, San Diego, California
• 2: Mark Sanderson , C. J. Van Rijsbergen, The impact on retrieval effectiveness of skewed frequency distributions, ACM Transactions on Information Systems (TOIS), v.17 n.4, p.440-465, Oct. 1999 [doi>10.1145/326440.326447]
• 3: H. Schutze and J. Pederson. Information retrieval based on word senses. In Proceedings of the 4th Annual Symposium on Document Analysis and Information Retrieval, pages 161--175, 1995.
• 4: I. Soboroff. Overview of the trec 2004 novelty track. In Proceedings of the Thirteenth Text REtrieval Conference (TREC 2004), NIST Special Publication 500--261, 2004.
• 5: E. Voorhees. Overview of the trec 2003 robust retrieval track. In Proceedings of the Twelfth Text REtrieval Conference Proceedings (TREC 2003), NIST Special Publication 500--255, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

General Terms:
Algorithms, Experimentation

Keywords:
ODP, ambiguity, quantification, rank correlation test