Term clustering of syntactic phrases

ABSTRACT

Term clustering and syntactic phrase formation are methods for transforming natural language text. Both have had only mixed success as strategies for improving the quality of text representations for document retrieval. Since the strengths of these methods are complementary, we have explored combining them to produce superior representations. In this paper we discuss our implementation of a syntactic phrase generator, as well as our preliminary experiments with producing phrase clusters. These experiments show small improvements in retrieval effectiveness resulting from the use of phrase clusters, but it is clear that corpora much larger than standard information retrieval test collections will be required to thoroughly evaluate the use of this technique.

REFERENCES

• 1: Michael It. Anderberg. Cluster Analysis for Applications. Academic Press, New York, 1973.
• 2: G. Antoniadis , G. Lallich-Boidin , Y. Polity , J. Rouault, A french text recognition model for information retrieval system, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.67-84, May 1988, Grenoble, France [doi>10.1145/62437.62441]
• 3: Bran Boguraev , Ted Briscoe, Large lexicons for natural language processing: utilising the grammar coding system of LDOCE, Computational Linguistics, v.13 n.3-4, July-December 1987
• 4: Chris Buckley , Alan F. Lewit, Optimization of inverted vector searches, Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval, p.97-110, June 05-07, 1985, Montreal, Quebec, Canada [doi>10.1145/253495.253515]
• 5: W. Bruce Croft. Clustering large files of documents using the aingle-link method. Journal of the American Society for Information Science, pages 341-344, November 1977.
• 6: W. B. Croft , R. Das, Experiments with query acquisition and use in document retrieval systems, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.349-368, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98240]
• 7: C. J. Crouch, A cluster-based approach to thesaurus construction, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.309-320, May 1988, Grenoble, France [doi>10.1145/62437.62467]
• 8: Martin Dillon and Ann S. Gray. FASIT" A fully automatic syntactically based indexing system. Journal of the American Society for Information Science, 34(2)'99-108, March 1983.
• 9: Joel L. Fagan. E#perim.ents irt Automatic Phrase l#dezing for Oocurnen# Retrieval: A Comparison of Syntactic and Non-Syntactic )tlethods. PhD thesis, Department of Computer Science, Cornel/University, September 1987.
• 10: Joel L. Fagan, The effectiveness of a nonsyntatic approach to automatic phrase indexing for document retrieval, Journal of the American Society for Information Science, v.40 n.2, p.115-132, March 1989 [doi>10.1002/(SICI)1097-4571(198903)40:2<115::AID-ASI6>3.0.CO;2-B]
• 11: E. A. Fox , G. L. Nunn , W. C. Lee, Coefficients of combining concept classes in a collection, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.291-307, May 1988, Grenoble, France [doi>10.1145/62437.62465]
• 12: Philip J. Hayes , Laura E. Knecht , Monica J. Cellio, A news story categorization system, Proceedings of the second conference on Applied natural language processing, February 09-12, 1988, Austin, Texas [doi>10.3115/974235.974238]
• 13: Donald J. Iiillman and Andrew J. Kasarda. The LEADER retrieval system. In AFIPS Proceedings 54#, pages 44 7-455, 1969.
• 14: Lynette Ilirschman, Ralph Grishman, and Naomi Sager. Grammatically-based automatic word class formation, lnformagion Processing and Management, 11:39-57, 1975.
• 15: J. Kittler. Feature selection and extraction. In Tzay Y. Young and King-Sun Fu, editors, Handbook of Pattern Recognition and Image Processing, pages 59-83. Academic Press, Orlando, 1986.
• 16: Paul H. Klingbiel. Machine-aided indexing of technical literature. Information Storage and Retrieval, 9:79-84, 1973.
• 17: R. Krovetz , W. B. Croft, Word sense disambiguation using machine-readable dictionaries, Proceedings of the 12th annual international ACM SIGIR conference on Research and development in information retrieval, p.127-136, June 25-28, 1989, Cambridge, Massachusetts, United States
• 18: M. E. Lesk. Word-word associations in document retrieval systems. American Documentation, pages 27-38, January 1969.
• 19: David D. Lewis. A description of CACM-3204-ML!, a test collection for information retrieval and machine learning. Information RetrievM Laboratory Memo 90-1, Computer and Information Science Department, University of Massachusetts at Amherst, 1990.
• 20: David Dolan Lewis, Representation and learning in information retrieval, University of Massachusetts, Amherst, MA, 1992
• 21: David D. Lewis, W. Bruce Croft, and Nehru Bhandaru. Language-oriented information retrieval. International Journal of Intelligent Systems, 4(3):285-318, 1989.
• 22: Karen E. Lochbaum , Lynn A. Streeter, Comparing and combining the effectiveness of latent semantic indexing and the ordinary vector space model for information retrieval, Information Processing and Management: an International Journal, v.25 n.6, p.665-676, 1989 [doi>10.1016/0306-4573(89)90100-3]
• 23: Jack Minker, Gerald A. Wilson, and Barbara H. Zi#nmerman. An evaluation of query expansion by tile addition of clustered terms for a document retrieval system. Information Storage and Retrieval, 8:329-348, 1972.
• 24: Paul M. Mort, David L. Waltz, Howard L. Resnikoff, and George G. Robertson. Automatic indexing of text. Ted,nical Report 86-1, Tltiltking btaehines Corporation, January 1986.
• 25: T. Noreault and It. Chatham. A procedure for the estimation of term similarity coefficients. Information Technology, pages 189-196, 1982.
• 26: M. F. Porter. An algoritlxrn for suffax stripping. Program, 14(3):130--137, July 1980.
• 27: G. Salton , M. E. Lesk, Computer Evaluation of Indexing and Text Processing, Journal of the ACM (JACM), v.15 n.1, p.8-36, Jan. 1968 [doi>10.1145/321439.321441]
• 28: Gerard. Salton, Automatic Information Organization and Retrieval., McGraw Hill Text, 1968
• 29: G. Salton , M. Smith, On the application of syntactic methodologies in automatic text analysis, Proceedings of the 12th annual international ACM SIGIR conference on Research and development in information retrieval, p.137-150, June 25-28, 1989, Cambridge, Massachusetts, United States
• 30: Tengku Moltd Tengku Sembok. Logical-Linguistic Model and Ezperirnents in Document Retrieval. PhD thesis, Department of Computing Science, University of Glasgow, August 1989.
• 31: A. F. Smeaton , C. J. van Rijsbergen, Experiments on incorporating syntactic processing of user queries into a document retrieval strategy, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.31-51, May 1988, Grenoble, France [doi>10.1145/62437.62439]
• 32: K. Sparck Jones #nd E. O. Barber. What makes an automatic keyword classification effective? Journal of the American Society for Information Science, pages 166--175, May-june 1971.
• 33: K. Sparck Jones and R. G. Bates. Research on automatic indexing 1974- 1976 (2 volumes). Technical report, Computer Laboratory. University of Cambridge, 1977.
• 34: K. Sparck Jones and j. I. Tait. Automatic search term variant generation. Journal of Documentation, 40(1):50-66, March 1984.
• 35: Karen Sparck Jones. Automatic Keyu,ord Classification for Ir#forma/io# Rctrie#,al. Archon Books, 1971.
• 36: Karen Sparck Jones. Collection properties influencing automatic term classification performance. Information Storage and Retrieval, 9:499-513, 1973.
• 37: G. Thurmair, A common architecture for different text processing techniques in an information retrieval environment, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.138-143, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253198]
• 38: H. Turtle , W. B. Croft, Inference networks for document retrieval, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.1-24, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98006]
• 39: Peter Willett. A fast procedure for the calculation of Similarity coefficients in automatic classification. Information Processing and Manager#ent, 17:53-60, 1981.
• 40: Clement T. Yu and Vijay V. Rnghavan. Single-pns.# method for determining the semantic relationships between terms. Journal of the American Society for Information Science, pages 345-354, November 197"7.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods; Abstracting methods

General Terms:
Experimentation, Languages