Query-specific clustering of search results based on document-context similarity scores

ABSTRACT

This paper presents a pilot study of query-specific clustering that uses our novel document-context based similarity scores as compared with document similarity scores. Clustering is applied to the top 1000 retrieved documents for a given query. Clustering effectiveness is evaluated based on the MK1 score for TREC-2, TREC-6 and TREC-7 test collections. Encouraging results were obtained whereby document-context clustering produces better MK1 scores than document clustering with a 95% confidence level if precision and recall are equally important.

REFERENCES

• 1: Anastasios Tombros , Robert Villa , C. J. Van Rijsbergen, The effectiveness of query-specific hierarchic clustering in information retrieval, Information Processing and Management: an International Journal, v.38 n.4, p.559-582, July 2002 [doi>10.1016/S0306-4573(01)00048-6]
• 2: D.A. Evans, J. Bennett, J. Montgomery, V. Sheftel, D.A. Hull and J.G. Shanahan. TREC-2004 HARD-track experiments in clustering. Proc. TREC Conference, 2004.
• 3: S. E. Robertson , S. Walker, Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.232-241, July 03-06, 1994, Dublin, Ireland
• 4: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• 5: H. C. Wu , R. W. P. Luk , K. F. Wong , K. L. Kwok , W. J. Li, A retrospective study of probabilistic context-based retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076179]
• 6: Gerard Salton , Edward A. Fox , Harry Wu, Extended Boolean information retrieval, Communications of the ACM, v.26 n.11, p.1022-1036, Nov. 1983 [doi>10.1145/182.358466]
• 7: Y. K. Kong, R.W.P. Luk, W. Lam, K.S. Ho and F.L. Chung. Passage-based retrieval based on parameterized fuzzy operators, ACM SIGIR Workshop on Mathematical/Formal Methods for Information Retrieval, 2004.
• 8: D. Harman. Private Communication at NTCIR-4, 2004.
• 9: N. Jardine and C.J.Van Rijsbergen. The use of hierarchical clustering in information retrieval. Information Storage and Retrieval, 7, 217--240, 1971.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

General Terms:
Experimentation, Performance

Keywords:
context-based model, document clustering, experimentations