A failure analysis of the limitation of suffixing in an online environment

ABSTRACT

The interaction of suffixing algorithms and ranking techniques in retrieval performance, particularly in an online environment, was investigated. Three general purpose suffixing algorithms were used for retrieval on the Cranfield 1400, Medlars, and CACM collections, and the results analysed with several standard evaluation measures. An examination of the retrieval performance using suffixing suggested two modifications to ranking techniques: variable weighting of word variants and selective stemming depending on query length. The experimental data is presented, and the limitations of suffixing in an online environment is discussed.

REFERENCES

• BELL79: Bell C. and Jones K.P.,"Toward Everyday Language Information Retrieval Systems via Minicomputers", Journal of the American Society for information Science, Vol. 30, No. 5, November 1979, pp. 334-338.
• BENSON86: Benson D.,Goldstein C.M., Fitzpatrick L., Williamson D. and Huntzinger R., "Developing Tools for Online Medical Reference Works", ProceeAings of Medinfo 86, Elsevier Science Publishers B.V., Amsterdam. 1986,, pp 558-560.
• CROFT83: Croft W.B., "Experiments with Representation in a Document Retrieval System", Information Technology: Research and Development 2 (1983), pp. 1 -21.
• HAFER74: Hafer M. and Weiss S., "Word Segmentation by Letter Successor Varieties", Information Storage and Retrieval 10, 1974, pp. 371-385.
• HARMAN86: Donna Williamson Harman, An experimental study of factors important in document ranking, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.186-193, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253206]
• LENNON81: Lennon M., Peirce D., Tarry B. and Willett P., "An Evaluation of Some Conflation Algorithms for Information RetrievaI", Journal of Information Science 3, 1981, pp. 177-188.
• LOVINS68: Lovins J.B., "Development of a Stemming Algorithm", Mechanical Translation and Computational Linguistics I 1, March 1968, pp. 22-31.
• PACAK78: Pacak M.G. and Pratt A.W., "Identification and Transformation of Terminal Morphemes in Medical English, Part II", Methods of Information in Medicine 17, 1978, pp 95-100.
• PORTER80: Porter M.F., "An Algorithm for Suffix Stripping", Program 14, Vol. 14, July 1980,pp. 130-137.
• RIJSBERGEN79: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• SALTON68: G. Salton, The SMART Retrieval System—Experiments in Automatic Document Processing, Prentice-Hall, Inc., Upper Saddle River, NJ, 1971
• SALTON83: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• SPARCK-JONES72: Sparck-Jones, K.,"A Statistical Interpretation of Term Sw.cificity and Its Aoolication in Retrieval", Journal of Documentation, Vol. 28, No. 1, March 1972, pp. 11-20.
• ULMSCHNEIDER83: Ulmschneider J. and Doszkocs T., "A Practical Stemming Algorithm for Online Search Assistance", Online Review 7(4), 1983, pp 301-315.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  C. Computer Systems Organization

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

General Terms:
Algorithms, Performance, Reliability