Query reformulation, search performance, and term suggestion devices in question-answering tasks

ABSTRACT

Capturing context within query in query reformulation tasks has been identified as a promising technique for supporting users who are engaged with interactive information retrieval systems. User queries represent the evolution of information problems. A deeper understanding of the structure and process of query reformulation, in particular, could provide further information for system adaptations. The present study characterizes the query reformulation process in two types of term suggestion devices, relevance feedback (RF) and Local Context Analysis (LCA), in simulated question-answering tasks using Rutgers' TREC-8 Interactive Track dataset. Four types of query reformulation were identified on the basis of semantic contents and relations, as well as sequences in users' modifications of queries. We found a significant relationship between the types of query reformulations and the use of term suggestion devices. But we did not find significant correlations between types of query reformulations and search performance. Some issues regarding systematic biases in query reformulations and capturing context within queries in interactive IR system are discussed.

REFERENCES

• 1: James Allan, Relevance feedback with too much data, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.337-343, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215380]
• 2: Jing Bai , Jian-Yun Nie , Guihong Cao , Hugues Bouchard, Using query contexts in information retrieval, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277747]
• 3: Bates, M. J. Information search tactics. Journal of the American Society for Information Science, 30, 4 (1979), 205--214.
• 4: Beaulieu, M. Interaction in information searching and retrieval. Journal of Documentation, 56, 4 (2000), 431--439.
• 5: N. J. Belkin , D. Kelly , G. Kim , J.-Y. Kim , H.-J. Lee , G. Muresan , M.-C. Tang , X.-J. Yuan , C. Cool, Query length in interactive information retrieval, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860474]
• 6: Belkin, N. J., Cool, C., Head, J., Jeng, J., Kelly, D., Lin, S., Lobash, L., Park, S. Y., Savage- Knepshield, P. and Sikora, C. Relevance feedback versus local context analysis as term suggestion devices: Rutgers' TREC-8 Interactive Track experience. In Information Technology: Eighth Text REtrieval Conference (TREC-8). Gaithersburg, MD: NIST, 2000, 565--73.
• 7: Belkin, N. J., Cool, C., Stein, A. and Thiel, U. Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems. Expert Systems with Applications, 9, 3 (1995), 379--395.
• 8: Belkin, N. J., Perez Carballo, J., Cool, C., Kelly, D., Lin, S., Park, S. Y., Rieh, S. Y., Savage Knepshield, P. & Sikora, C. Rutgers' TREC-7 interactive track experience. In TREC-7. Proceedings of the Seventh Text REtrieval Conference. Washington, DC: NIST, 1999, 275--283.
• 9: Blair, D. C. Searching biases in large interactive document retrieval systems. Journal of the American Society for Information Science, 31, 4 (1980), 271--277.
• 10: Callan, J. P., Croft, W. B. & Harding, S. M. The INQUERY retrieval system. In Dexa 3, Proceedings of the Third International Conference on Database and Expert System Applications. Berlin: Springer Verlag, (1992), 78--83.
• 11: Efthimis N. Efthimiadis, Interactive query expansion: a user-based evaluation in a relevance feedback environment, Journal of the American Society for Information Science, v.51 n.11, p.989-1003, Sept. 2000 [doi>10.1002/1097-4571(2000)9999:9999<::AID-ASI1002>3.0.CO;2-B]
• 12: Fidel, R. Searchers' selection of search keys: III. Searching styles. Journal of the American Society for Information Science, 42, 7 (1991), 515--527.
• 13: Jane Greenberg, Automatic query expansion via lexical-semantic relationships, Journal of the American Society for Information Science and Technology, v.52 n.5, p.402-415, March 2001 [doi>10.1002/1532-2890(2001)9999:9999<::AID-ASI1089>3.3.CO;2-B]
• 14: Hersh, W. and Over, P. TREC-8 interactive track report. Proceedings of the Eighth Text REtrieval Conference (TREC-8). (16--19 Nov. 1999). NIST, Gaithersburg, MD, USA, 2000, 57--64.
• 15: Joho, H., Sanderson, M. and Beaulieu, M. Study of user interaction with a concept-based interactive query expansion support tool. Lecture Notes in Computer Science, 2997 (2004), 42--56.
• 16: Lancaster, F. W. MEDLARS: Report on the evaluation of its operating efficiency. American Documentation, 20, 2 (1969), 119--142.
• 17: Muhr, T. Atlas. ti, The Knowledge Workbench (Version 4.2) {Computer software}. Berlin: Scientific Software Development, (1997).
• 18: Soo Young Rieh , Hong Xie, Analysis of multiple query reformulations on the web: the interactive information retrieval context, Information Processing and Management: an International Journal, v.42 n.3, p.751-768, May 2006 [doi>10.1016/j.ipm.2005.05.005]
• 19: S. E. Robertson , M. M. Hancock-Beaulieu, On the evaluation of IR systems, Information Processing and Management: an International Journal, v.28 n.4, p.457-466, July-Aug. 1992 [doi>10.1016/0306-4573(92)90004-J]
• 20: Ian Ruthven, Re-examining the potential effectiveness of interactive query expansion, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860475]
• 21: Salton, G. and Buckley, C. Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41, 4 (1990), 288--97.
• 22: G. Salton , M. E. Lesk, Computer Evaluation of Indexing and Text Processing, Journal of the ACM (JACM), v.15 n.1, p.8-36, Jan. 1968 [doi>10.1145/321439.321441]
• 23: Saracevic, T. The stratified model of information retrieval interaction: Extension and applications. Proceedings of the ASIS Annual Meeting, 34 (1997), 313--327.
• 24: Springer, E. V. and Tang, R. A communication perspective on meta-search engine query structure: A pilot study. In CoLIS 4: Proceedings of the Fourth International Conference on Conceptions of Library and Information Science. Libraries Unlimited, Greenwood Village, CO, 2002, 323--326.
• 25: Padmini Srinivasan, Query expansion and MEDLINE, Information Processing and Management: an International Journal, v.32 n.4, p.431-443, July 1996 [doi>10.1016/0306-4573(95)00076-3]
• 26: Lucy A. Suchman, Plans and situated actions: the problem of human-machine communication, Cambridge University Press, New York, NY, 1987
• 27: Voorhees, E. M. and Harman, D. Proceedings of the Eighth Text REtrieval Conference (TREC-8). NIST, Gaithersburg, MD, 2000.
• 28: Ellen M. Voorhees, Query expansion using lexical-semantic relations, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.61-69, July 03-06, 1994, Dublin, Ireland
• 29: Jinxi Xu , W. Bruce Croft, Query expansion using local and global document analysis, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.4-11, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243202]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Keywords:
context within query, query expansion, query reformulation, question-answering tasks