The impact on retrieval effectiveness of skewed frequency distributions

ABSTRACT

We present an analysis of word senses that provides a fresh insight into the impact of word ambiguity on retrieval effectiveness with potential broader implications for other processes of information retrieval. Using a methodology of forming artifically ambiguous words, known as pseudowords, and through reference to other researchers' work, the analysis illustrates that the distribution of the frequency of occurrance of the senses of a word plays a strong role in ambiguity's impact of effectiveness. Further investigation shows that this analysis may also be applicable to other processes of retrieval, such as Cross Language Information Retrieval, query expansion, retrieval of OCR'ed texts, and stemming. The analysis appears to provide a means of explaining, at least in part, reasons for the processes' impact (or lack of it) on effectiveness.

REFERENCES

• 1: Lisa Ballesteros , W. Bruce Croft, Phrasal translation and query expansion techniques for cross-language information retrieval, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.84-91, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 2: BURNETT,J.E.,COOPER, D., LYNCH,M.F.,WILLETT, P., AND WYCHERLEY, M. 1979. Document retrieval experiments using indexing vocabularies of varying size. -1. Variety generation symbols assigned to the fronts of index terms. J. Doc. 35, 3, 197-206.
• 3: Kenneth Ward Church, One term or two?, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.310-318, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215376]
• 4: CRESTANI, F., SANDERSON, M., THEOPHYLACTOU, M., AND LALMAS, M. 1997. Short queries, natural language and spoken documents retrieval: Experiments at Glasgow University. In Proceedings of the 6th Text Retreival Conference (TREC-6, Nov.), E. Voorhees and D. Harman, Eds.
• 5: GALE, W., CHURCH,K.W.,AND YAROWSKY, D. 1992a. Work on statistical methods for word sense disambiguation. In Intelligent Probabilistic Approaches to Natural Language Papers from the 1992 Fall Symposium. AAAI Press, Menlo Park, CA, 54-60.
• 6: William A. Gale , Kenneth W. Church , David Yarowsky, One sense per discourse, Proceedings of the workshop on Speech and Natural Language, February 23-26, 1992, Harriman, New York [doi>10.3115/1075527.1075579]
• 7: William Gale , Kenneth Ward Church , David Yarowsky, Estimating upper and lower bounds on the performance of word-sense disambiguation programs, Proceedings of the 30th annual meeting on Association for Computational Linguistics, p.249-256, June 28-July 02, 1992, Newark, Delaware [doi>10.3115/981967.981999]
• 8: Gregory Grefenstette, Explorations in Automatic Thesaurus Discovery, Kluwer Academic Publishers, Norwell, MA, 1994
• 9: D. Harman, A failure analysis of the limitation of suffixing in an online environment, Proceedings of the 10th annual international ACM SIGIR conference on Research and development in information retrieval, p.102-107, June 03-05, 1987, New Orleans, Louisiana, United States [doi>10.1145/42005.42017]
• 10: Donna Harman, Ranking algorithms, Information retrieval: data structures and algorithms, Prentice-Hall, Inc., Upper Saddle River, NJ, 1992
• 11: David A. Hull , Gregory Grefenstette, Querying across languages: a dictionary-based approach to multilingual information retrieval, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.49-57, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243212]
• 12: KILGARRIFF, A. 1997. I don't believe in word senses. Comput. Hum. 31, 2, 91-113.
• 13: Robert Krovetz, Viewing morphology as an inference process, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.191-202, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160718]
• 14: Robert Krovetz , W. Bruce Croft, Lexical ambiguity and information retrieval, ACM Transactions on Information Systems (TOIS), v.10 n.2, p.115-141, April 1992 [doi>10.1145/146802.146810]
• 15: Michael Lesk, Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone, Proceedings of the 5th annual international conference on Systems documentation, p.24-26, June 1986, Toronto, Ontario, Canada [doi>10.1145/318723.318728]
• 16: David Dolan Lewis, Representation and learning in information retrieval, University of Massachusetts, Amherst, MA, 1992
• 17: George A. Miller, WordNet: a lexical database for English, Communications of the ACM, v.38 n.11, p.39-41, Nov. 1995 [doi>10.1145/219717.219748]
• 18: Hwee Tou Ng , Hian Beng Lee, Integrating multiple knowledge sources to disambiguate word sense: an exemplar-based approach, Proceedings of the 34th annual meeting on Association for Computational Linguistics, p.40-47, June 24-27, 1996, Santa Cruz, California [doi>10.3115/981863.981869]
• 19: PORTER, M. F. 1980. An algorithm for suffix stripping. Program: Autom. Libr. Inf. Syst. 14,3, 130-137.
• 20: Gerard Salton , Edward A. Fox , Harry Wu, Extended Boolean information retrieval, Communications of the ACM, v.26 n.11, p.1022-1036, Nov. 1983 [doi>10.1145/182.358466]
• 21: Mark Sanderson, Word sense disambiguation and information retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.142-151, July 03-06, 1994, Dublin, Ireland
• 22: SANDERSON, M. 1996. Word sense disambiguaiton and information retrieval. Tech. Rep. TR-1997-7. Deparment of Computing Science, University of Glasgow, Glasgow, UK.
• 23: SCH~TZE, H. 1992. Context space. In Intelligent Probabilistic Approaches to Natural Language Papers from the 1992 Fall Symposium. AAAI Press, Menlo Park, CA, 113-120.
• 24: SCH~TZE,H.AND PEDERSEN, J. O. 1995. Information retrieval based on word senses. In Symposium on Document Analysis and Information Retrieval (Las Vegas, NV). 161-175.
• 25: SMALL,S.AND RIEGER, C. 1982. Parsing and comprehending with word experts (a theory and its realisation). In Strategies for Natural Language Processing, W. G. Lehnert and M. H. Ringle, Eds. 89-148.
• 26: Alan F. Smeaton , Ian Quigley, Experiments on using semantic distances between words in image caption retrieval, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.174-180, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243261]
• 27: Alan F. Smeaton , A. Lawrence Spitz, Using Character Shape Coding for Information Retrieval, Proceedings of the 4th International Conference on Document Analysis and Recognition, p.974, August 18-20, 1997
• 28: SPARCK JONES,K.AND VAN RIJSBERGEN, C. J. 1976. Progress in documentation. J. Doc. 32,1 (Mar.), 59-75.
• 29: Michael Sussna, Word sense disambiguation for free-text indexing using a massive semantic network, Proceedings of the second international conference on Information and knowledge management, p.67-74, November 01-05, 1993, Washington, D.C., United States [doi>10.1145/170088.170106]
• 30: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• 31: Ellen M. Voorhees, Using WordNet to disambiguate word senses for text retrieval, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.171-180, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160715]
• 32: Ellen M. Voorhees, Query expansion using lexical-semantic relations, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.61-69, July 03-06, 1994, Dublin, Ireland
• 33: VOORHEES,E.AND HARMAN, D. 1996. Overview of the Fifth Text REtrieval Conference (TREC-5). In Proceedings of the 5th Text Retrieval Conference (TREC-5, Gaithersburg, MD, Nov.), E. M. Voorhees and D. K. Harman, Eds. National Institute of Standards and Technology, Gaithersburg, MD.
• 34: WALLIS, P. 1993. Information retrieval based on paraphrase. In Proceedings of the 1st PACLING Conference.
• 35: WEISS, S. F. 1973. Learning to disambiguate. Inf. Storage Retrieval 9, 33-41.
• 36: WILKS, Y., FASS, D., GUO, C., MACDONALD,J.E.,PLATE, T., AND SLATOR, B. M. 1990. Providing machine tractable dictionary tools. Mach. Transl. 5, 2, 99-154.
• 37: Jinxi Xu , W. Bruce Croft, Corpus-based stemming using cooccurrence of word variants, ACM Transactions on Information Systems (TOIS), v.16 n.1, p.61-81, Jan. 1998 [doi>10.1145/267954.267957]
• 38: David Yarowsky, One sense per collocation, Proceedings of the workshop on Human Language Technology, March 21-24, 1993, Princeton, New Jersey [doi>10.3115/1075671.1075731]
• 39: ZIPF, G. K. 1949. Human Behavior and Principle of Least Effort: An Introduction to Human Ecology. Addison-Wesley, Reading, MA.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis
  I.6 SIMULATION AND MODELING

General Terms:
Experimentation, Measurement

Keywords:
pseudowords, word sense ambiguity, word sense disambiguation