Measuring the informativeness of a retrieval process

ABSTRACT

Evaluation of information retrieval systems should be based on measures of the information provided by the retrieval process, “informativeness” measures which take into account the interactive and full-text nature of present-day systems and the different types of questions which are asked of them. Desirable properties for an informativeness measure are developed, including context sensitivity, user centrality, and logarithmic response. A hypergraph-based framework for measuring the informativeness of a retrieval process is presented and a measure developed which satisfies the desired properties. The measure is compared to previously developed information measures and illustrated via an application.

REFERENCES

• 1: Bar-Hilel, Y. and Carnap, R., "Semantic information, " British Journal for the Philosophy of Science 4(13) , 1953, 147-57.
• 2: Belkin, N., "Information concepts for information science," Journal of Documentation 34(1), 1978, 55- 85.
• 3: Buckland, M., "Information as Thing," Journal of the American Society for Information Science 42(5), 1991, 351-60.
• 4: Eisenberg, M. and Barry, C., "Order effects- a study of the possible influence of presentation order on user judgments of document relevance." Journal of the American Society for Information Science 39(5), 1988, 293-300.
• 5: Fox, C.J., Information and Misinformation: an Investiqation of the Notions of information, Misinformation. Informinq. and Misinforming, Greenwood Pr., 1983.
• 6: Fritz Machlup , Una Mansfield, The study of information: interdisciplinary messages, John Wiley & Sons, Inc., New York, NY, 1983
• 7: May, K.O., "Growth and quality of the mathematical literature," ISIS 59(199), winter, 1968, 363-371.
• 8: Howard L. Resnikoff, The illusion of reality, Springer-Verlag New York, Inc., New York, NY, 1989
• 9: Claude E. Shannon , Warren Weaver, A Mathematical Theory of Communication, University of Illinois Press, Champaign, IL, 1963
• 10: Shreider, Y., "On the semantic characteristics of information," Information Storaqe and Retrieval 2(4), 1965, 221-233.
• 11: J. Tague , R. Schultz, Evaluation of the user interface in an information retrieval system: a model, Information Processing and Management: an International Journal, v.25 n.4, p.377-389, 1989 [doi>10.1016/0306-4573(89)90066-6]
• 12: Jean Tague , Airi Salminen , Charles McClellan, Complete formal model for information retrieval systems, Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, p.14-20, October 13-16, 1991, Chicago, Illinois, United States [doi>10.1145/122860.122862]
• 13: Wells, R. "A measure of subjective information, " in Structure of Languaqe and Its Mathematical Aspects; Proceedings of Svmposia in Applied Mathematics 12, 1961, 237-244.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  G. Mathematics of Computing
  G.2 DISCRETE MATHEMATICS
      G.2.2 Graph Theory
          Subjects: Graph algorithms

  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

General Terms:
Algorithms, Design, Human Factors, Measurement