A comparison of query and term suggestion features for interactive searching

ABSTRACT

Query formulation is one of the most difficult and important aspects of information seeking and retrieval. Two techniques, term relevance feedback and query suggestion, provide methods to help users formulate queries, but each is limited in different ways. In this research we combine these two techniques by automatically creating query suggestions using term relevance feedback techniques. To evaluate our approach, we conducted an interactive information retrieval study with 55 subjects and 20 topics. Each subject completed four topics, half with a term suggestion system and half with a query suggestion system. We also investigated the source of the suggestions: approximately half of all subjects were provided with system-generated suggestions, while half were provided with user-generated suggestions. Results show that subjects used more query suggestions than term suggestions and saved more documents with these suggestions, even though there were no significant differences in performance. Subjects preferred the query suggestion system and rated it higher along a number of dimensions including its ability to help them think of new approaches to searching. Qualitative data provided insight into subjects' usage and ratings, and indicated that subjects often used the suggestions even when they did not click on them.

REFERENCES

• 1: Beaulieu, M.&Jones, S. (1998). Interactive searching and interface issues in the Okapi best match probabilistic retrieval system. Interacting with Computers, 10, 237--248.
• 2: Doug Beeferman , Adam Berger, Agglomerative clustering of a search engine query log, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.407-416, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347176]
• 3: N. J. Belkin , C. Cool , D. Kelly , S.-J. Lin , S. Y. Park , J. Perez-Carballo , C. Sikora, Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval, Information Processing and Management: an International Journal, v.37 n.3, p.403-434, May 2001 [doi>10.1016/S0306-4573(00)00055-8]
• 4: Bodo Billerbeck , Falk Scholer , Hugh E. Williams , Justin Zobel, Query expansion using associated queries, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956866]
• 5: Larry Fitzpatrick , Mei Dent, Automatic feedback using past queries: social searching?, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.306-313, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 6: Jill Freyne , Rosta Farzan , Peter Brusilovsky , Barry Smyth , Maurice Coyle, Collecting community wisdom: integrating social search & social navigation, Proceedings of the 12th international conference on Intelligent user interfaces, January 28-31, 2007, Honolulu, Hawaii, USA [doi>10.1145/1216295.1216312]
• 7: Hsieh-Yee, I. (1993).Effects of search experience and subject knowledge on online search behavior: Measuring the search tactics of novice and experienced searchers. Journal of the American Society for Information Science 44, 161--174.
• 8: Kalervo Järvelin , Jaana Kekäläinen, Cumulated gain-based evaluation of IR techniques, ACM Transactions on Information Systems (TOIS), v.20 n.4, p.422-446, October 2002 [doi>10.1145/582415.582418]
• 9: Järvelin, K., Price, S. L, Delcambre, L. M. L.,&Nielsen, M. L. (2008). Discounted cumulated gain based evaluation of multiple-query IR sessions. Proceedings of ECIR '08, Glasgow, Scotland.
• 10: Hideo Joho , Claire Coverson , Mark Sanderson , Micheline Beaulieu, Hierarchical presentation of expansion terms, Proceedings of the 2002 ACM symposium on Applied computing, March 11-14, 2002, Madrid, Spain [doi>10.1145/508791.508916]
• 11: Rosie Jones , Benjamin Rey , Omid Madani , Wiley Greiner, Generating query substitutions, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135835]
• 12: Diane Kelly , Xin Fu, Elicitation of term relevance feedback: an investigation of term source and context, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148249]
• 13: Kelly, D., Gyllstrom, K.,&Bailey, E. W. (2009). Remote vs. face-to-face study mode: Differences in user behaviors, performance and feedback. SILS Tech. Report, 2009-02 (http://sils.unc.edu/research/techreports.html).
• 14: Qiaozhu Mei , Dengyong Zhou , Kenneth Church, Query suggestion using hitting time, Proceeding of the 17th ACM conference on Information and knowledge management, October 26-30, 2008, Napa Valley, California, USA [doi>10.1145/1458082.1458145]
• 15: Vijay V. Raghavan , Hayri Sever, On the reuse of past optimal queries, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.344-350, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215381]
• 16: Ian Ruthven, Re-examining the potential effectiveness of interactive query expansion, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860475]
• 17: Ian Ruthven , Mounia Lalmas, A survey on the use of relevance feedback for information access systems, The Knowledge Engineering Review, v.18 n.2, p.95-145, June 2003 [doi>10.1017/S0269888903000638]
• 18: Barry Smyth , Evelyn Balfe , Jill Freyne , Peter Briggs , Maurice Coyle , Oisin Boydell, Exploiting Query Repetition and Regularity in an Adaptive Community-Based Web Search Engine, User Modeling and User-Adapted Interaction, v.14 n.5, p.383-423, January 2005 [doi>10.1007/s11257-004-5270-4]
• 19: Ryen W. White , Mikhail Bilenko , Silviu Cucerzan, Studying the use of popular destinations to enhance web search interaction, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277771]
• 20: Ryen W. White , Gary Marchionini, Examining the effectiveness of real-time query expansion, Information Processing and Management: an International Journal, v.43 n.3, p.685-704, May, 2007 [doi>10.1016/j.ipm.2006.06.005]
• 21: Pertti Vakkari, Changes in Search Tactics and Relevance Judgements when Preparing a Research Proposal A Summary of the Findings of a Longitudinal Study, Information Retrieval, v.4 n.3-4, p.295-310, September-December 2001
• 22: Voorhees, E. M. (2006). Overview of the TREC 2005 Robust Retrieval Track. Proceedings of TREC-14.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Design, Human Factors, Performance

Keywords:
interactive searching, query suggestion, relevance feedback, term suggestion