User term feedback in interactive text-based image retrieval

ABSTRACT

To alleviate the vocabulary problem, this paper investigates the role of user term feedback in interactive text-based image retrieval. Term feedback refers to the feedback from a user on specific terms regarding their relevance to a target image. Previous studies have indicated the effectiveness of term feedback in interactive text retrieval [14]. However, the term feedback has not shown to be effective in our experiments on text-based image retrieval. Our results indicate that, although term feedback has a positive effect by allowing users to identify more relevant terms, it also has a strong negative effect by providing more opportunities for users to specify irrelevant terms. To understand these different effects and their implications on the potential of term feedback, this paper further presents analysis of important factors that contribute to the utility of term feedback and discusses the outlook of term feedback in interactive text-based image retrieval.

REFERENCES

• 1: Peter G. Anick , Suresh Tipirneni, The paraphrase search assistant: terminological feedback for iterative information seeking, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.153-159, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312670]
• 2: N. J. Belkin , P. G. Marchetti, Determining the functionality features of an intelligent interface to an information retrieval system, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.151-177, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98901]
• 3: Belkin, N.J., Cool, C., Koenemann, J., Ng, K.B., and Park, S. Using Relevance Feedback and Ranking in Interactive Searching. In Proceedings of TREC4. 1996.
• 4: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 5: Youngok Choi , Edie M. Rasmussen, Searching for images: the analysis of users' queries for image retrieval in American history, Journal of the American Society for Information Science and Technology, v.54 n.6, p.498-511, April 2003 [doi>10.1002/asi.10237]
• 6: Clough, P., Sanderson, M., and Reid, Norman. The Eurovision St Andrews Photographic Collection. http://ir.shef.ac.uk/imageclef2004/guide.pdf.
• 7: D. Harman, Towards interactive query expansion, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.321-331, May 1988, Grenoble, France [doi>10.1145/62437.62469]
• 8: He, X., King, O., Ma, W.-Y., Li, M., and Zhang, H. J. Learning a semantic space from user's relevance feedback for image retrieval. IEEE Transaction on Curcuits and Systems for Video Technology, 13(1):39--48, Jan. 2003.
• 9: Hearst, M. Using Categories to Provide Context for Full-Text Retrieval Results. In Proceedings of RIAO'94. 1994.
• 10: Scott Henninger , Nicholas J. Belkin, Interface issues and interaction strategies for information retrieval systems, Conference companion on Human factors in computing systems: common ground, p.352-353, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/257089.257364]
• 11: Chu-Hong Hoi , Michael R. Lyu, A novel log-based relevance feedback technique in content-based image retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027533]
• 12: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 13: Keister, L. H. User types and queries: impact on image access systems. In Challenges in indexing electronic text and images (Fidel, R et al., eds). ASIS, 1994, 7--22.
• 14: Jürgen Koenemann , Nicholas J. Belkin, A case for interaction: a study of interactive information retrieval behavior and effectiveness, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.205-212, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238487]
• 15: John Lafferty , Chengxiang Zhai, Document language models, query models, and risk minimization for information retrieval, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.111-119, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383970]
• 16: Victor Lavrenko , W. Bruce Croft, Relevance based language models, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.120-127, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383972]
• 17: Lavrenko, V., Manmatha, R., and Jeon, J. A Model for Learning the Semantics of Pictures. In Proceedings of Advance in Neutral Information Processing. 2003.
• 18: Jia Li , James Z. Wang, Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1075-1088, September 2003 [doi>10.1109/TPAMI.2003.1227984]
• 19: Sharon McDonald , John Tait, Search strategies in content-based image retrieval, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860452]
• 20: Florent Monay , Daniel Gatica-Perez, On image auto-annotation with latent space models, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957070]
• 21: Jay Michael Ponte, A language modeling approach to information retrieval, University of Massachusetts, Amherst, MA, 1998
• 22: Rui, Y., Huang, T. S., Ortega, M., and Mehrotra, S. Relevance Feedback: A Power Tool in Interactive Content-Based Image Retrieval, IEEE Trans. on Circuits and Systems for Video Technology, Special Issue on Segmentation, Description, and Retrieval of Video Content, pp644--655, Vol 8, No. 5, Sept, 1998.
• 23: Exploratory image databases: content-based retrieval, Academic Press, Inc., Duluth, MN, 2001
• 24: Jeroen Vendrig , Marcl Worring , Arnold W. M. Smeulders, Filter Image Browsing: Interactive Image Retrieval by Using Database Overviews, Multimedia Tools and Applications, v.15 n.1, p.83-103, September 2001 [doi>10.1023/A:1011367820253]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process; Relevance feedback
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Experimentation, Human Factors, Performance

Keywords:
interactive image retrieval, text-based image retrieval, user term feedback