User-oriented document clustering: a framework for learning in information retrieval

ABSTRACT

In information retrieval, cluster analysis is an important tool employed to enhance both efficiency and effectiveness of the retrieval process. Most clustering algorithms have difficulty in reflecting the closeness of documents as perceived by the user. A two phase scheme for document clustering, whose results reflect the “conceptual” clusters that are perceived by the user of the retrieval system, is proposed. Since the clusters obtained by this scheme are not characterized in terms of the document representations, a strategy for cluster searching is also developed. Both the proposed document clustering scheme and document searching strategy are experimentally evaluated using a test collection from the SMART system. The preliminary experimental results obtained are very encouraging.

REFERENCES

• 1: Doyle, L.B. (1966). Breaking the Cost Barrier in Aut#atlc Classification. Report No. SP-2516, System Development Corp., Santa Nonl ca, Ca.
• 2: Dattola, R.T. (1971). Experiments with a fast algorithm for automatic classification. In the SMART Retrieval System- Experiments in Automatic Document Processing, G. Salton, Ed., Prentice-Hall, Englewood Cliffs, N.J.
• 3: Rocchlo, J.J. Jr. (1966). #nt retrieval systems - optimization and evaluation. Doctoral thesis. In: Information Storage and Retrieval, Scientific Report NO. ISR-10, Harvard Univ., Cambridge, MaSs.
• 4: van Rijsbergen, C.J. and Croft, W.B. (1975). Document clustering: An evaluation of some experln#ts with the Cranfield 1400 collectlon. Inf. Process #anage., 11, 171-182.
• 5: G. Salton , A. Wong, Generation and search of clustered files, ACM Transactions on Database Systems (TODS), v.3 n.4, p.321-346, Dec. 1978 [doi>10.1145/320289.320291]
• 6: Yu, C.T. (1974). A clustering algorlthm . based on user queries. J. Amer. Soc. Infom. Sol., 25, 218-226.
• 7: C. T. Yu , Y. T. Wang , C. H. Chen, Adaptive document clustering, Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval, p.197-203, June 05-07, 1985, Montreal, Quebec, Canada [doi>10.1145/253495.253525]
• 8: Sparck-Jc#es, K. (1971). Automatic Keyword Classiflcatlon for Information Retrieval. Archon Books, Cormecticut.
• 9: S. Kumar, Semantic Clustering of Index Terms, Journal of the ACM (JACM), v.15 n.4, p.493-513, Oct. 1968 [doi>10.1145/321479.321480]
• 10: Dattola, R.T. and Ruffly, O.M. (1967). An experiment in autcmatlc thesaurus ccnstructlon. In: Information Storage and Retrieval, Scientific Report No. ISR-13, Chpt. VIII, Cornell University, Ithaca, N.Y.
• 11: Clement T. Yu, A Formal Construction of Term Classes, Journal of the ACM (JACM), v.22 n.1, p.17-37, Jan. 1975 [doi>10.1145/321864.321867]
• 12: Vijay V. Raghavan , C. T. Yu, Experiments on the determination of the relationships between terms, ACM Transactions on Database Systems (TODS), v.4 n.2, p.240-260, June 1979 [doi>10.1145/320071.320081]
• 13: Jardine, N. and Sibson, R. (1968). Amodel{ for taxonomy. Mathematical Biosciences, 2, 456-482.
• 14: Ellen M. Voorhees, The cluster hypothesis revisited, Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval, p.188-196, June 05-07, 1985, Montreal, Quebec, Canada [doi>10.1145/253495.253524]
• 15: Jardlne, N. and van Rijsbergen, C.J. (1971). The use of hierarchic clustering in information retrieval. Information Storage and Retrieval, 7,217-240.
• 16: Rocchio, Jr., J.O. (1971). Relevance feedback in information retrieval, in the Smart Retrieval System- Experiments in Autcmatlc Document Processing, G. Salton, Ed., Prentice Hall Inc., Engl# Cliffs, N.J.
• 17: Knuth, D.E. (1973). Sorting and Searching- The Art of #ter Progrmsmlng. Vol. 3, Addison-Wesley Publishing Co., Reading, N#.
• 18: Salton, G. (1980). Automatic term class cormtructlon using relevance -A summary of work in autum#tlc psmxlo-classlfication. Inf. Process. and Nanage., 16,1-15.
• 19: Robertson, S.E. and Sparck Jo#es, K. (1976). Relevance welghtlng of search terms. J. of AmDr. Soc. Inform. Sci., 27, 129-146.
• 20: C. T. Yu , G. Salton, Precision Weighting—An Effective Automatic Indexing Method, Journal of the ACM (JACM), v.23 n.1, p.76-88, Jan. 1976 [doi>10.1145/321921.321930]
• 21: C. T. Yu , W. S. Luk, Analysis of Effectiveness of Retrieval in Clustered Files, Journal of the ACM (JACM), v.24 n.4, p.607-622, Oct. 1977 [doi>10.1145/322033.322039]
• 22: C. T. Yu , W. S. Luk , M. K. Siu, On the estimation of the number of desired records with respect to a given query, ACM Transactions on Database Systems (TODS), v.3 n.1, p.41-56, March 1978 [doi>10.1145/320241.320245]
• 23: Croft, W.B. (1980). A model of cluster searching based on classification. Information systems, 5,189-195.