Re-examining the potential effectiveness of interactive query expansion

ABSTRACT

Much attention has been paid to the relative effectiveness of interactive query expansion versus automatic query expansion. Although interactive query expansion has the potential to be an effective means of improving a search, in this paper we show that, on average, human searchers are less likely than systems to make good expansion decisions. To enable good expansion decisions, searchers must have adequate instructions on how to use interactive query expansion functionalities. We show that simple instructions on using interactive query expansion do not necessarily help searchers make good expansion decisions and discuss difficulties found in making query expansion decisions.

REFERENCES

• 1: Beaulieu, M. Experiments with interfaces to support query expansion. Journal of Documentation. 53. 1. pp 8--19. 1997.
• 2: Dorothee Blocks , Ceri Binding , Daniel Cunliffe , Douglas Tudhope, Qualitative Evaluation of Thesaurus-Based Retrieval, Proceedings of the 6th European Conference on Research and Advanced Technology for Digital Libraries, p.346-361, September 16-18, 2002
• 3: Chang, Y. K., Cirillo, C., and Razon, J. Evaluation of feedback retrieval using modified freezing, residual collection & test and control groups. The SMART retrieval system - experiments in automatic document processing. G. Salton (ed). Chapter 17. pp 355--370. 1971.
• 4: Efthimis N. Efthimiadis, User choices: a new yardstick for the evaluation of ranking algorithms for interactive query expansion, Information Processing and Management: an International Journal, v.31 n.4, p.605-620, July 1995 [doi>10.1016/0306-4573(95)00070-W]
• 5: Efthimiadis, E. N.. Query expansion. ARIST Volume 31: Annual Review of Information Science and Technology. Martha E. Williams (ed). 1996.
• 6: Fowkes, H., and Beaulieu, M. Interactive searching behaviour: Okapi experiment for TREC-8. Proceedings of IRSG 2000. 22nd Annual Colloquium on Information Retrieval Research. Cambridge. Cambridge. 2002.
• 7: D. Harman, Towards interactive query expansion, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.321-331, May 1988, Grenoble, France [doi>10.1145/62437.62469]
• 8: Mirja Iivonen, Consistency in the selection of search concepts and search terms, Information Processing and Management: an International Journal, v.31 n.2, p.173-190, March/April 1995 [doi>10.1016/0306-4573(94)00062-8]
• 9: Bernard J. Jansen , Amanda Spink , Tefko Saracevic, Real life, real users, and real needs: a study and analysis of user queries on the web, Information Processing and Management: an International Journal, v.36 n.2, p.207-227, Jan.1.2000 [doi>10.1016/S0306-4573(99)00056-4]
• 10: Jürgen Koenemann , Nicholas J. Belkin, A case for interaction: a study of interactive information retrieval behavior and effectiveness, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.205-212, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238487]
• 11: Magennis, M. The potential and actual effectiveness of interactive query expansion. PhD thesis. University of Glasgow. 1997.
• 12: Mark Magennis , Cornelis J. van Rijsbergen, The potential and actual effectiveness of interactive query expansion, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.324-332, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 13: S. E. Robertson, On term selection for query expansion, Journal of Documentation, v.46 n.4, p.359-364, Dec. 1990 [doi>10.1108/eb026866]
• 14: Robertson, S. E., and Sparck Jones, K. Relevance weighting of search terms. Journal of the American Society for Information Science. 27. 3. pp 129--146. 1976.
• 15: Amanda Spink , Tefko Saracevic, Interaction in information retrieval: selection and effectiveness of search terms, Journal of the American Society for Information Science, v.48 n.8, p.741-761, Aug. 1997 [doi>10.1002/(SICI)1097-4571(199708)48:8<741::AID-ASI7>3.0.CO;2-S]
• 16: Ellen M. Voorhees , Donna Harman, Overview of the sixth text REtrieval conference (TREC-6), Information Processing and Management: an International Journal, v.36 n.1, p.3-35, Jan. 2000 [doi>10.1016/S0306-4573(99)00043-6]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Experimentation, Human Factors

Keywords:
evaluation, query expansion