A very fast prototype retrieval system using statistical ranking

ABSTRACT

The most striking result in working with Professor Gerald Salton over 20 years ago on the comparison between the SMART system and the MEDLARS system [Salton69] was the fact that whereas Boolean retrieval (MEDLARS) did very well or very poorly, the SMART system always seemed to find some of the relevant records. All of us working at Cornell University during that time wanted to run a full-scale comparison between these systems to demonstrate what was to us the clear superiority of a ranking retrieval system, but this was not possible due to lack of funding and other problems.

REFERENCES

• 1: {Cooper76} Cooper William S., "The Paradoxical Role of Unexamined Documents in the Evaluation of Retrieval Effectiveness", Information Processing and Management, Vol. 12, 1976, pp. 367--375.
• 2: Tamas E. Doszkocs, From research to application: the cite natural language information retrieval system, Proceedings of the 5th annual ACM conference on Research and development in information retrieval, p.251-262, May 18-20, 1982, West Berlin, Germany
• 3: Donna Williamson Harman, An experimental study of factors important in document ranking, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.186-193, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253206]
• 4: {Harman88a} Harman D., Benson D., Fitzpatrick L., Huntzinger R. and Goldstein C., "IRX: An Information Retrieval System for Experimentation and User Applications", Proceedings of RIAO '88, Boston, 1988, pp. 840--848.
• 5: D. Harman, Towards interactive query expansion, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.321-331, May 1988, Grenoble, France [doi>10.1145/62437.62469]
• 6: {Knuth73} Knuth Donald E., The Art of Computer Programming, Addison-Wesley Publishing Company, Reading, Massachusetts, 1973, pp. 325--326.
• 7: {Lefkovitz75} Lefkovitz David, "The Large Data Base File Structure Dilemma", Journal of Chemical Information and Computer Sciences, Vol. 15, No. 1, 1975, pp. 14--19.
• 8: {Salton69} Salton G., "A Comparison between Manual and Automatic Indexing Methods", American Documentation, Vol. 20, No. 1, January 1969, pp. 61--71.
• 9: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 10: {Sparck Jones72} Sparck Jones K., "A Statistical Interpretation of Term Specificity and Its Application in Retrieval", Journal of Documentation, Vol. 28, No. 1, March 1972, pp. 11--20.
• 11: Jean Tague , Paul Nicholls, The maximal value of a Zipf size variable: sampling properties and relationship to other parameters, Information Processing and Management: an International Journal, v.23 n.3, p.155-170, 1987 [doi>10.1016/0306-4573(87)90001-X]
• 12: {Van Rijsbergen76} van Rijsbergen C. J., "File Organization in Library Automation and Information Retrieval", Journal of Documentation, Vol. 32, No. 4, December 1976, pp. 294--317.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  F. Theory of Computation
  F.2 ANALYSIS OF ALGORITHMS AND PROBLEM COMPLEXITY
      F.2.2 Nonnumerical Algorithms and Problems
          Subjects: Sorting and searching

  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical computing

General Terms:
Design, Experimentation, Management, Measurement, Performance, Theory