Multi-model similarity propagation and its application for web image retrieval

ABSTRACT

In this paper, we propose an iterative similarity propagation approach to explore the inter-relationships between Web images and their textual annotations for image retrieval. By considering Web images as one type of objects, their surrounding texts as another type, and constructing the links structure between them via webpage analysis, we can iteratively reinforce the similarities between images. The basic idea is that if two objects of the same type are both related to one object of another type, these two objects are similar; likewise, if two objects of the same type are related to two different, but similar objects of another type, then to some extent, these two objects are also similar. The goal of our method is to fully exploit the mutual reinforcement between images and their textual annotations. Our experiments based on 10,628 images crawled from the Web show that our proposed approach can significantly improve Web image retrieval performance.

REFERENCES

• 1: Antonini, M., Barlaud, M., Mathieu, P., and Daubchies, I., Image Coding Using Wavelet Transform, IEEE Trans. on Image Processing. 1(2). (Apr. 1992). 205--220.
• 2: Barnard, K., Forsyth, D. Learning the Semantic of Words and Pictures. ICCV (2001).
• 3: Barnard, K., Duygulu, P., and Forsyth, D. Clustering Art. CVPR (2001), 434--439
• 4: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 5: Cai, D., Yu, S.P., Wen, J.R., and W.-Y. Ma, Extracting Content Structure for Web Pages Based on Visual Representation, APWeb (2003), 406--417
• 6: Cai, D., Yu, S.P., Wen, J.R., and W.-Y. Ma, VIPS: a Vision-Based Page Segmentation Algorithm, Microsoft Technical Report (2003), MSR-TR-2003-79
• 7: M. La Cascia , S. Sethi , S. Sclaroff, Combining Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web, Proceedings of the IEEE Workshop on Content - Based Access of Image and Video Libraries, p.24, June 21-21, 1998
• 8: Zheng Chen , Liu Wenyin , Feng Zhang , Mingjing Li, Web mining for web image retrieval, Journal of the American Society for Information Science and Technology, v.52 n.10, p.831-839, August 2001 [doi>10.1002/asi.1132.abs]
• 9: Duffing, G. Text-Image Interaction for Image Retrieval and Semi-Automatic Indexing. 20th Annual BCS-IRSG Colloquium on IR, (1998)
• 10: David Gibson , Jon Kleinberg , Prabhakar Raghavan, Inferring Web communities from link topology, Proceedings of the ninth ACM conference on Hypertext and hypermedia : links, objects, time and space---structure in hypermedia systems: links, objects, time and space---structure in hypermedia systems, p.225-234, June 20-24, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276627.276652]
• 11: Jing Huang , S. Ravi Kumar , Mandar Mitra , Wei-Jing Zhu , Ramin Zabih, Image Indexing Using Color Correlograms, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.762, June 17-19, 1997
• 12: Kandola, J., Shawe-Taylor, J., Cristianini, N. Learning Semantic Similarity. NIPS (2002)
• 13: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Proceedings of the ninth annual ACM-SIAM symposium on Discrete algorithms, p.668-677, January 25-27, 1998, San Francisco, California, United States
• 14: Neville, J., and Jensen, D. Iterative Classification in Relational Data. Proceedings of the AAAI 2000 Workshop, AAAI Press. (2000). 42--49.
• 15: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
• 16: Seán Slattery , Mark Craven, Combining Statistical and Relational Methods for Learning in Hypertext Domains, Proceedings of the 8th International Workshop on Inductive Logic Programming, p.38-52, July 22-24, 1998
• 17: Srihari, R.K., Rao, A.B., Han, B., Munirathnam, S., and Wu, X.Y. A Model For Multimodal Information Retrieval. ICME (2000)/.
• 18: Stricker, M., and Orengo, M. Similarity of Color Images, In Storage and Retrieval for Image and Video Databases III, SPIE 2420, (Feb. 1995), 381--392
• 19: Jidong Wang , Huajun Zeng , Zheng Chen , Hongjun Lu , Li Tao , Wei-Ying Ma, ReCoM: reinforcement clustering of multi-type interrelated data objects, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860486]
• 20: Thijs Westerveld, Probabilistic multimedia retrieval, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564482]
• 21: Gui-Rong Xue , Hua-Jun Zeng , Zheng Chen , Wei-Ying Ma , Yong Yu, Similarity spreading: a unified framework for similarity calculation of interrelated objects, Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, May 19-21, 2004, New York, NY, USA [doi>10.1145/1013367.1013525]
• 22: Xiang Sean Zhou , Thomas S. Huang, Unifying Keywords and Visual Contents in Image Retrieval, IEEE MultiMedia, v.9 n.2, p.23-33, April 2002 [doi>10.1109/93.998050]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Design, Performance

Keywords:
iterative similarity propagation, mixture model, multimedia retrieval, mutual reinforcement