Similarity space projection for web image search and annotation

ABSTRACT

Web image search has been explored and developed in academic as well as commercial areas for over a decade. To measure the similarity between Web images and user queries, most of the existing Web image search systems try to convert an image to textual keywords by analyzing the textual information available (such as surrounding text and image filename) with or without leveraging image visual features (such as color, texture, shape). In this way, the existing systems transform "Web images" to the "query (text)" space so as to compare the relevance of images to the query. In this paper, we present a novel solution to Web image search - similarity space projection (SSP). This algorithm takes images and queries as two heterogeneous object peers, and projects them into a third Euclidean "similarity space" in which their similarity can be directly measured. The rule of projection guarantees that in the new space the relevant images are kept close to the corresponding query and those irrelevant ones are away from it. Experiments on real-world Web image collections showed that the proposed algorithm significantly outperformed traditional information retrieval models (such as vector space model) in the application of image search. Besides Web image search, we demonstrate that this algorithm can also be applied to image annotation scenario, and has promising performance. Thus, this algorithm unifies Web image search and image annotation into same framework.

REFERENCES

• 1: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 2: Stephen Boyd , Lieven Vandenberghe, Convex Optimization, Cambridge University Press, New York, NY, 2004
• 3: Deng Cai , Xiaofei He , Zhiwei Li , Wei-Ying Ma , Ji-Rong Wen, Hierarchical clustering of WWW image search results using visual, textual and link information, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027747]
• 4: Charles Frankel , Michael J Swain , Vassilis Athitsos, WebSeer: An Image Search Engine for the World Wide Web, University of Chicago, Chicago, IL, 1996
• 5: Cheng Thao, Ethan V.Munson. A relevance model for Web image search. WDA2003, UK, August 3, 2003.
• 6: Kobus Barnard , Pinar Duygulu , David Forsyth , Nando de Freitas , David M. Blei , Michael I. Jordan, Matching words and pictures, The Journal of Machine Learning Research, 3, 3/1/2003
• 7: Huamin Feng , Rui Shi , Tat-Seng Chua, A bootstrapping framework for annotating and retrieving WWW images, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027748]
• 8: Jing Huang , S. Ravi Kumar , Mandar Mitra , Wei-Jing Zhu , Ramin Zabih, Image Indexing Using Color Correlograms, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.762, June 17-19, 1997
• 9: Michael Kearns , Dana Ron, Algorithmic stability and sanity-check bounds for leave-one-out cross-validation, Neural Computation, v.11 n.6, p.1427-1453, Aug. 15, 1999 [doi>10.1162/089976699300016304]
• 10: Wei-Hao Lin , Rong Jin , Alexander Hauptmann, Web Image Retrieval Re-Ranking with Relevance Model, Proceedings of the IEEE/WIC International Conference on Web Intelligence, p.242, October 13-17, 2003
• 11: Ye Lu , Chunhui Hu , Xingquan Zhu , HongJiang Zhang , Qiang Yang, A unified framework for semantics and feature based relevance feedback in image retrieval systems, Proceedings of the eighth ACM international conference on Multimedia, p.31-37, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354403]
• 12: Stan Sclaroff , Marco La Cascia , Saratendu Sethi, Unifying textual and visual cues for content-based image retrieval on the World Wide Web, Computer Vision and Image Understanding, v.75 n.1-2, p.86-98, July/Aug. 1999 [doi>10.1006/cviu.1999.0765]
• 13: Singhal, A.. Modern information retrieval: A brief overview. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 24(4):35--43, 2001.
• 14: John R. Smith , Shih-Fu Chang, Visually Searching the Web for Content, IEEE MultiMedia, v.4 n.3, p.12-20, July 1997 [doi>10.1109/93.621578]
• 15: Steven M. Beitzel , Eric C. Jensen , Abdur Chowdhury , David Grossman , Ophir Frieder, Hourly analysis of a very large topically categorized web query log, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009048]
• 16: Tsymbalenko, Y., and Munson, E.V. Using HTML metadata to find relevant images on the Web. Proc. of Internet Computing 2001, Vol.II, pp.842--848, June 2001.
• 17: Yanai, K. Image collector II: an over-one-thousand-image-gathering system. WWW2003, Budapest Hungary, 2003.
• 18: Yanai, K. Web image mining toward generic image recognition. WWW, May 2003.
• 19: Yu H., Li M., Zhang H. and Feng J. Color texture moment for content-based image retrieval. ICIP, September, 2002.
• 20: Zhang L., Lin F. and Zhang B. A CBIR method based on color-spatial feature. TENCON'99, pp166--169, 1999.
• 21: Altavista image: http://www.altavista.com/images
• 22: Ditto: http://ditto.com/
• 23: Google image search: http://images.google.com
• 24: http://www.google.com/press/zeitgeist.html
• 25: Yahoo image search: http://images.yahoo.com
• 26: PicSearch: http://www.picsearch.com

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Algorithms, Design, Experimentation, Performance, Theory

Keywords:
image annotation, similarity space projection, web image search