Contextual relevance feedback

ABSTRACT

We present results of a preliminary study of a pile-based information retrieval interface that supports contextual relevance feedback. We designed two interfaces based on the pile metaphor, one which supported contextual relevance feedback and the other which did not, and conducted a within-subjects laboratory evaluation with 24 subjects. Results demonstrate support for the pile-based approach generally, with users indicating that it assisted them in structuring and managing their search results. There were significant differences between usability assessments of each interface and in the interactions in which subjects engaged while they were seeking information. Although we found no significant differences in performance between interfaces, subjects showed a strong preference for the contextual relevance feedback interface over the other, and believed that they did better with that interface.

REFERENCES

• 1: Allan, J. (2006). HARD Track overview in TREC 2005 high accuracy retrieval from documents. In E. M. Voorhees & L. P. Buckland (Eds.), TREC-2005, Proceedings of the Fourteenth Text Retrieval Conference. Washington, D.C.: GPO.
• 2: Michelle Q. Wang Baldonado , Terry Winograd, SenseMaker: an information-exploration interface supporting the contextual evolution of a user's interests, Proceedings of the SIGCHI conference on Human factors in computing systems, p.11-18, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258563]
• 3: Beaulieu, M. (1997). Experiments on interfaces to support query expansion. Journal of Documentation, 53(1), 8--19.
• 4: N. J. Belkin , C. Cool , D. Kelly , S.-J. Lin , S. Y. Park , J. Perez-Carballo , C. Sikora, Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval, Information Processing and Management: an International Journal, v.37 n.3, p.403-434, May 2001 [doi>10.1016/S0306-4573(00)00055-8]
• 5: Olha Bondarenko , Ruud Janssen, Documents at Hand: Learning from Paper to Improve Digital Technologies, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1054990]
• 6: Borlund, P., & Ingwersen, P. (2000). The development of a method for evaluating interactive information retrieval systems. Journal of Documentation, 53(3), 225--250.
• 7: Campbell, I. & van Rijsbergen, C. J. (1996). The ostensive model of developing information needs. Proceedings. of Conceptions of Library and Information Science (CoLIS-2), 251--268.
• 8: Paul Dourish, What we talk about when we talk about context, Personal and Ubiquitous Computing, v.8 n.1, p.19-30, February 2004 [doi>10.1007/s00779-003-0253-8]
• 9: Dumais, S. T., & Belkin, N. J. (2005). The TREC Interactive Tracks: Putting the user into search. In E. M. Voorhees & D. K. Harman (Eds.) TREC: Experiment and Evaluation in Information Retrieval (pp. 123--153), Cambridge, MA: MIT Press.
• 10: Efthimis N. Efthimiadis, Interactive query expansion: a user-based evaluation in a relevance feedback environment, Journal of the American Society for Information Science, v.51 n.11, p.989-1003, Sept. 2000 [doi>10.1002/1097-4571(2000)9999:9999<::AID-ASI1002>3.0.CO;2-B]
• 11: D. Harman, Towards interactive query expansion, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.321-331, May 1988, Grenoble, France [doi>10.1145/62437.62469]
• 12: Marti A. Hearst , Jan O. Pedersen, Reexamining the cluster hypothesis: scatter/gather on retrieval results, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.76-84, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243216]
• 13: David G. Hendry , David J. Harper, An informal information-seeking environment, Journal of the American Society for Information Science, v.48 n.11, p.1036-1048, Nov. 1997 [doi>10.1002/(SICI)1097-4571(199711)48:11<1036::AID-ASI6>3.3.CO;2-E]
• 14: Hersh, W. & Over, P. (2001a). Introduction to a special issue on interactivity at the Text Retrieval Conference (TREC). Information Processing & Management 37(3), 365--367.
• 15: Hersh, W., & Over, P. (1999). TREC-8 interactive track report. In D. Harman and E. M. Voorhees (Eds.), The Eighth Text Retrieval Conference (TREC-8), 57--64.
• 16: Peter Ingwersen , Kalervo Järvelin, The Turn: Integration of Information Seeking and Retrieval in Context (The Information Retrieval Series), Springer-Verlag New York, Inc., Secaucus, NJ, 2005
• 17: Kelly, D., Diaz, F., Belkin, N. J., & Allan, J. (2004). A user-centered approach to evaluating topic models. Proceedings of the European Conference on Information Retrieval (ECIR '04), Sunderland, UK., 27--41.
• 18: Jürgen Koenemann , Nicholas J. Belkin, A case for interaction: a study of interactive information retrieval behavior and effectiveness, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.205-212, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238487]
• 19: Lund, A. M. (2001). Measuring usability with the USE Questionnaire. Usability and User Experience, 8(2). Available online: http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html.
• 20: Richard Mander , Gitta Salomon , Yin Yin Wong, A “pile” metaphor for supporting casual organization of information, Proceedings of the SIGCHI conference on Human factors in computing systems, p.627-634, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.143055]
• 21: Martin, I., & Jose, J. M. (2004). Fetch: A personalised information retrieval tool. Proceedings of RIAO '04, 405--419.
• 22: Gheorghe Muresan , David J. Harper, Topic modeling for mediated access to very large document collections, Journal of the American Society for Information Science and Technology, v.55 n.10, p.892-910, August 2004 [doi>10.1002/asi.20034]
• 23: Daniel E. Rose , Richard Mander , Tim Oren , Dulce B. Poncéleon , Gitt Salomon , Yin Yin Wong, Content awareness in a file system interface: implementing the “pile” metaphor for organizing information, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.260-269, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160735]
• 24: Ian Ruthven, Re-examining the potential effectiveness of interactive query expansion, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860475]
• 25: Saracevic, T. (1997). The stratified model of information retrieval interaction: Extension and applications. Proceedings of the Annual Meeting of the American Society for Information Science (ASIS '97), USA, 3--9.
• 26: Russell C. Swan , James Allan, Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.173-181, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290987]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Experimentation, Human Factors, Measurement

Keywords:
IR interfaces, context, interactive IR, relevance feedback, user study