System scoring using partial prior information

ABSTRACT

We introduce smoothing of retrieval effectiveness scores, which balances results from prior incomplete query sets against limited additional complete information, in order to obtain more refined system orderings than would be possible on the new queries alone.

REFERENCES

• 1: Stanley F. Chen , Joshua Goodman, An empirical study of smoothing techniques for language modeling, Proceedings of the 34th annual meeting on Association for Computational Linguistics, p.310-318, June 24-27, 1996, Santa Cruz, California [doi>10.3115/981863.981904]
• 2: Ellen M. Voorhees, The TREC robust retrieval track, ACM SIGIR Forum, v.39 n.1, June 2005 [doi>10.1145/1067268.1067272]
• 3: William Webber , Alistair Moffat , Justin Zobel, Score standardization for inter-collection comparison of retrieval systems, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390346]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)

General Terms:
Experimentation, Measurement, Performance

Keywords:
partial information, score aggregation, smoothing, system ordering