Scalable relevance feedback using click-through data for web image retrieval

ABSTRACT

Relevance feedback (RF) has been extensively studied in the content-based image retrieval community. However, no commercial Web image search engines support RF because of scalability, efficiency and effectiveness issues. In this paper we proposed a scalable relevance feedback mechanism using click-through data for web image retrieval. The proposed mechanism regards users' click-through data as implicit feedback which could be collected at lower cost, in larger quantities and without extra burden on the user. During RF process, both textual feature and visual feature are used in a sequential way. To seamlessly combine textual feature-based RF and visual feature-based RF, a query concept-dependent fusion strategy is automatically learned. Experimental results on a database consisting of nearly three million Web images show that the proposed mechanism is wieldy, scalable and effective.

REFERENCES

• 1: E. Cheng et al., "Using Implicit Relevance Feedback to Advance Image Search," To appear in Proc. of ICME 2006.
• 2: F. Jing et al., "A Unified Framework for Image Retrieval Using Keyword and Visual Features," IEEE Trans. on Image Processing, 14(7): 979--89, 2005.
• 3: J. Rocchio, Relevance Feedback in Information Retrieval. Prentice-Hall, 1971.
• 4: Lei Zhang , Yuxiao Hu , Mingjing Li , Weiying Ma , Hongjiang Zhang, Efficient propagation for face annotation in family albums, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027689]
• 5: Qiankun Zhao , Steven C. H. Hoi , Tie-Yan Liu , Sourav S. Bhowmick , Michael R. Lyu , Wei-Ying Ma, Time-dependent semantic similarity measure of queries using historical click-through data, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135858]
• 6: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 7: Stan Sclaroff , Leonid Taycher , Marco La Cascia, ImageRover: A Content-Based Image Browser for the World Wide Web, Proceedings of the 1997 Workshop on Content-Based Access of Image and Video Libraries (CBAIVL '97), p.2, June 20-20, 1997
• 8: Thorsten Joachims , Laura Granka , Bing Pan , Helene Hembrooke , Geri Gay, Accurately interpreting clickthrough data as implicit feedback, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076063]
• 9: Till Quack , Ullrich Mönich , Lars Thiele , B. S. Manjunath, Cortina: a system for large-scale, content-based web image retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027650]
• 10: X.S. Zhou et al., "Relevance Feedback in Image Retrieval: A Comparative Study," ACM Multimedia Systems, 8(6):536--544, 2003.
• 11: Ye Lu , Chunhui Hu , Xingquan Zhu , HongJiang Zhang , Qiang Yang, A unified framework for semantics and feature based relevance feedback in image retrieval systems, Proceedings of the eighth ACM international conference on Multimedia, p.31-37, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354403]
• 12: Y. Rui et al. "Relevance feedback: A Power Tool for Interactive Content-based Image Retrieval," IEEE Trans. on CSVT, 13(4):811--820, 1998.
• 13: Yi Wu , Edward Y. Chang , Kevin Chen-Chuan Chang , John R. Smith, Optimal multimodal fusion for multimedia data analysis, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027665]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Algorithms, Experimentation, Performance

Keywords:
click-through, implicit feedback, relevance feedback, web image retrieval