Ambiguous requests: implications for retrieval tests, systems and theories

ABSTRACT

Retrieval system experimentation has assumed that user requests represent a single information need. The problem is identifying and meeting this need. Search engine experience demonstrates that this assumption is far from holding in the real world. Responding appropriately to this fact raises new issues for research on retrieval system theory, design, and evaluation.

REFERENCES

• 1: Andrei Broder, A taxonomy of web search, ACM SIGIR Forum, v.36 n.2, Fall 2002 [doi>10.1145/792550.792552]
• 2: Jaime Carbonell , Jade Goldstein, The use of MMR, diversity-based reranking for reordering documents and producing summaries, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.335-336, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291025]
• 3: Harr Chen , David R. Karger, Less is more: probabilistic models for retrieving fewer relevant documents, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148245]
• 4: W. Bruce Croft, Advances in Informational Retrieval: Recent Research from the Center for Intelligent Information Retrieval, Kluwer Academic Publishers, Norwell, MA, 2000
• 5: K. Sparck Jones , S. Walker , S. E. Robertson, A probabilistic model of information retrieval: development and comparative experiments Part 2, Information Processing and Management: an International Journal, v.36 n.6, p.809-840, Nov.06.2000 [doi>10.1016/S0306-4573(00)00016-9]
• 6: Voorhees, E. M. 'Question answering in TREC', in TREC. Experiment and evaluation in information retrieval, Ed. E. M. Voorhees and D. K. Harman, Cambridge, MA: MITP Press, 2005, 233--257.
• 7: Robertson, S. E. 'The probability ranking principle in IR', Journal of Documentation, 33, 1977, 130--137.
• 8: Ellen M. Voorhees , Donna K. Harman, TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing), The MIT Press, 2005
• 9: Cheng Xiang Zhai , William W. Cohen , John Lafferty, Beyond independent relevance: methods and evaluation metrics for subtopic retrieval, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860440]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  F. Theory of Computation
  F.2 ANALYSIS OF ALGORITHMS AND PROBLEM COMPLEXITY
      F.2.2 Nonnumerical Algorithms and Problems
          Subjects: Sorting and searching

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software

General Terms:
Algorithms, Design, Documentation