The cluster hypothesis revisited

ABSTRACT

A new means of evaluating the cluster hypothesis is introduced and the results of such an evaluation are presented for four collections. The results of retrieval experiments comparing a sequential search, a cluster-based search, and a search of the clustered collection in which individual documents are scored against the query are also presented. These results indicate that while the absolute performance of a search on a particular collection is dependent on the pairwise similarity of the relevant documents, the relative effectiveness of clustered retrieval versus sequential retrieval is independent of this factor. However, retrieval of entire clusters in response to a query usually results in a poorer performance than retrieval of individual documents from clusters.

REFERENCES

• 1: Salton, G., ed., (1971) The SMART Retrieval System. Prentice-Hall, Englewood Cliffs, N.J.
• 2: Marcia Davis Kerchner, Dynamic document processing in clustered collections, 1971
• 3: Daniel Mcclure Murray, Document retrieval based on clustered files, 1972
• 4: Williamson, R.E., (1974) Real-time Document Retrieval. Ph.D. Thesis, Cornell University.
• 5: Jardine, N. and van Rijsbergen, C. J., (1971) The Use of Hierarchic Clustering in Information Retrieval. Inform. Stor. ~ Retr., 7, 217- 240.
• 6: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• 7: Ide, Eleanor Rose Cook, (1969) Relevance Feedback in an Automatic Document Retrieval System. Master Thesis, Cornell University. Report ISR-15 to the National Science Foundation.
• 8: Edward Alan Fox, Extending the boolean and vector space models of information retrieval with p-norm queries and multiple concept types, 1983
• 9: Gerard Salton , Edward A. Fox , Harry Wu, Extended Boolean information retrieval, Communications of the ACM, v.26 n.11, p.1022-1036, Nov. 1983 [doi>10.1145/182.358466]
• 10: van Rijsbergen, C. J. and Sparck Jones, K., (1973) A Test for the Separation of Relevant and Non-relevant Documents in Experimental Retrieval Collections. Journal of Documentation, 29, 251-257.
• 11: van Rijsbergen, C. J., (1974) Further Experiments with Hierarchic Clustering in Document Retrieval. Inform. Stor. ~ Retr., }tO, 1-14.
• 12: van Rijsbergen, C. J. and Croft, W. B., (1975) Document Clustering: An Evaluation of Some Experiments with the Cranfield 1400 Collection. Inform. Proc. ~ Mana#ement, 11, 171- 182.
• 13: Croft, W. B., (1980) A Model of Cluster Searching Based on Classification. Inform. Systems, 5, 189--195.
• 14: Griffiths, Alan, Robinson, Lesley A., and Willett, Peter, (1984) Hierarchic Agglomerative Clustering Methods for Automatic Document Classification. Journal of Documentation, 40, 175-205.