Using WordNet to disambiguate word senses for text retrieval

ABSTRACT

This paper describes an automatic indexing procedure that uses the “IS-A” relations contained within WordNet and the set of nouns contained in a text to select a sense for each plysemous noun in the text. The result of the indexing procedure is a vector in which some of the terms represent word senses instead of word stems. Retrieval experiments comparing the effectivenss of these sense-based vectors vs. stem-based vectors show the stem-based vectors to be superior overall, although the sense-based vectors do improve the performance of some queries. The overall degradation is due in large part to the difficulty of disambiguating senses in short query statements. An analysis of these results suggests two conclusions: the IS-A links define a generalization/specialization hierarchy that is not sufficient to reliably select the correct sense of a noun from the set of fine sense distinctions in WordNet; and missing correct matches because of incorrect sense resolution has a much more deleterious effect on retrieval performance than does making spurious matches.

REFERENCES

• 1: Eric Brill, A simple rule-based part of speech tagger, Proceedings of the third conference on Applied natural language processing, March 31-April 03, 1992, Trento, Italy [doi>10.3115/974499.974526]
• 2: Edward Alan Fox, Extending the boolean and vector space models of information retrieval with p-norm queries and multiple concept types, 1983
• 3: Robert Krovetz , W. Bruce Croft, Lexical ambiguity and information retrieval, ACM Transactions on Information Systems (TOIS), v.10 n.2, p.115-141, April 1992 [doi>10.1145/146802.146810]
• 4: George Miller. Special Issue, WordNet: An on-line lexical database. International Journal of Lexicography, 3(4)~ 1990.
• 5: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
• 6: Gerard Salton and Michael E. Lesk. Information analysis and dictionary construction. In Gerard Salton, editor, The SMART Retrieval System: Experiments in Automatic Document Processing, chapter 6, pages 115-142. Prentice-Hall, Inc. Englewood Cliffs, New Jersey, 1971.
• 7: Sally Yeates Sedelow and Donna Weir Mooney. Knowledge retrieval from domaintranscendent expert systems: II. research results. In Proceedings of the 51st Annual Meeting of the American Society of Information Science, pages 209-212, 1988.
• 8: Brian Michael Slator. Lexical Semantics and Preference Semantics Analysis. PhD thesis, New Mexico State University, Las Cruces, NM, December 1988.
• 9: Ellen M. Voorhees and Yuan-Wang Hou. Vector expansion in a large collection. In Proceedings of the First Text Retmeval Conference, 1992. Proceedings to appear.
• 10: Ellen M. Voorhees, Claudia Leacock, and Geoffrey Towell. Learning context to disambiguate word senses. In Proceedings of the 3rd Computational Learning Theory and Natural Learning Systems Conference, 1992. Proceedings to appear. Also available as Siemens technical report.
• 11: G.K. Zipf. The meaning-frequency relationship of words. Journal of General Psychology, 3:251-256, 1945.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Experimentation, Languages, Performance