An evaluation of phrasal and clustered representations on a text categorization task

ABSTRACT

Syntactic phrase indexing and term clustering have been widely explored as text representation techniques for text retrieval. In this paper we study the properties of phrasal and clustered indexing languages on a text categorization task, enabling us to study their properties in isolation from query interpretation issues. We show that optimal effectiveness occurs when using only a small proportion of the indexing terms available, and that effectiveness peaks at a higher feature set size and lower effectiveness level for a syntactic phrase indexing than for word-based indexing. We also present results suggesting that traditional term clustering method are unlikely to provide significantly improved text representations. An improved probabilistic text categorization method is also presented.

REFERENCES

• BFL+88: P. Biebricher , N. Fuhr , G. Lustig , M. Schwantner , G. Knorz, The automatic indexing system AIR/PHYS - from research to applications, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.333-342, May 1988, Grenoble, France [doi>10.1145/62437.62470]
• Chu88: Kenneth Ward Church, A stochastic parts program and noun phrase parser for unrestricted text, Proceedings of the second conference on Applied natural language processing, February 09-12, 1988, Austin, Texas [doi>10.3115/974235.974260]
• Cro83: Croft, W. Experiments with representation in a document retrieval system. Information Technology: Research and Development, 2:1-21, 1983.
• CD90: W. B. Croft , R. Das, Experiments with query acquisition and use in document retrieval systems, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.349-368, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98240]
• CTL91: W. Bruce Croft , Howard R. Turtle , David D. Lewis, The use of phrases and structured queries in information retrieval, Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, p.32-45, October 13-16, 1991, Chicago, Illinois, United States [doi>10.1145/122860.122864]
• Cro88: C. J. Crouch, A cluster-based approach to thesaurus construction, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.309-320, May 1988, Grenoble, France [doi>10.1145/62437.62467]
• DH73: Duda, R.. and Hart, P. Pattern Classification and Scene Analysis. Wiley- Interscience, New York, 1973.
• Fag87: Fagan, J. Ezperiments in Automatic Phrase Indezing for Document Retrieval: A Comparison of Syntactic and Non-Syntactic Methods. PhD thesis, Dept. of Computer Science, Cornell University, 1987.
• Fie75: Field, B. Towards automatic indexing: Automatic assignment of controlledlanguage indexing and classification from free indexing. Journal of Documentation, 31(4):246-265, December 1975.
• Fuh89: Norbert Fuhr, Models for retrieval with probabilistic indexing, Information Processing and Management: an International Journal, v.25 n.1, p.55-72, 1989 [doi>10.1016/0306-4573(89)90091-5]
• FB90: N. Fuhr , C. Buckley, Probabilistic document indexing from relevance feedback data, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.45-61, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98008]
• FHL+91: Fuhr, N., Hartmann, S., Lustig, G., Schwantner, M., Tzeras, K., and Knorz, G.. AII~/X~a rule-based multistage indexing system for large subject fields. RIAO 91, pp. 606-623, 1991.
• HZ80: Hamill, K. and Zamora, A. The use of titles for automatic document classification. JASIS, pp. 396-402, Nov. 1980.
• HW90: Philip J. Hayes , Steven P. Weinstein, CONSTRUE/TIS: A System for Content-Based Indexing of a Database of News Stories, Proceedings of the The Second Conference on Innovative Applications of Artificial Intelligence, p.49-64, May 01-03, 1990
• JC82: Jain, A. and Chandrasekaran, B. Dimensionality and sample size considerations in pattern recognition practice. In Krishnaiah, P. and Kanal, L., eds., Handbook of Statistics, Vol. Z, pp. 835-855. North-Holland, Amsterdam, 1982.
• Jam85: Mike James, Classification algorithms, Wiley-Interscience, New York, NY, 1985
• Lan86: Lancaster, F. Vocabulary Control for Information Retrieval. Information Resources, Arlington, VA, 2nd edition, 1986.
• LC90: D. D. Lewis , W. B. Croft, Term clustering of syntactic phrases, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.385-404, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98244]
• Lew91: David D. Lewis, Evaluating text categorization, Proceedings of the workshop on Speech and Natural Language, p.312-318, February 19-22, 1991, Pacific Grove, California [doi>10.3115/112405.112471]
• Lew92: David Dolan Lewis, Representation and learning in information retrieval, University of Massachusetts, Amherst, MA, 1992
• Lew92c: Lewis, D. and tkinguette, M. Text categorization by inductive learning. Manuscript, Jan. 1992.
• Mar61: M. E. Maron, Automatic Indexing: An Experimental Inquiry, Journal of the ACM (JACM), v.8 n.3, p.404-417, July 1961 [doi>10.1145/321075.321084]
• Mur83: Murtagh, F. A survey of recent advances in hierarchical clustering algorithms. The Computer Journal, 26(4) :354-359, 1983.
• PW91: Peat, H. and Willett, P. The limitations of term co-occurrence data for query expansion in document retrieval systems. JASIS, 42(5):378-383, 1991.
• RS76: l~obertson, S. and Sparck :Iones, K. l~elevance weighting of search terms. JASIS, pp. 129-146, May-June 1976.
• SB90: Salton, G. and Buckley, C. Improving retrieval performance by relevance feedback. JASIS, 41(4):288-297, 1990.
• SS91: Smeaton, A. and Sheridan, P. Using morpho-syntactic language analysis in phrase matching. RIAO-91, pp. 414- 429, 1991.
• Spa73a: Sparck Jones, K. Collection properties influencing automatic term classification performance. Information Storage and Retrieval, 9:499-513, 1973.
• Str92: Strzalkowski, T. Information retrieval using robust natural language processing. In Proceedings of Speech and Natural Language Workshop, Morgan Kaufmann: San Mateo, CA, Feb. 1992. To appear.
• Sun91: Sundheim, B., editor. Proceedings of the Third Message Understanding Evaluation and Conference, Morgan Kaufmann, Los Altos, CA, May 1991.
• van77: van Kijsbergen, C. A theoretical basis for the use of co-occurrence data in information retrieval. Journal of Documentation, 33(2):106-119, June 1977.
• van79: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• van81: van RJjsbergen, C. Retrieval effectiveness. In Sparck :Iones, K., editor, information Retrieval E~periment. Butterworths, London, 1981.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING

General Terms:
Algorithms, Languages